server/querier.py
author Rémi Cardona <remi.cardona@logilab.fr>
Mon, 14 Sep 2015 16:03:07 +0200
changeset 10612 84468b90e9c1
parent 10609 e2d8e81bfe68
child 10613 8d9fe02387e3
permissions -rw-r--r--
[py3k] basestring → six.string_types
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
10087
ed0b076c119b [rset] kill the rset._rqlst cache
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9990
diff changeset
     1
# copyright 2003-2014 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
"""Helper classes to execute RQL queries on a set of sources, performing
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
security checking and data aggregation.
5726
c3b99606644d [json] fix json serialization for recent simplejson implementation, and test encoding of entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5657
diff changeset
    20
"""
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
    21
from __future__ import print_function
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
    22
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
from itertools import repeat
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
10612
84468b90e9c1 [py3k] basestring → six.string_types
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10609
diff changeset
    27
from six import string_types
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    28
from six.moves import range
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    29
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    30
from rql import RQLSyntaxError, CoercionError
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
    31
from rql.stmts import Union
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
    32
from rql.nodes import ETYPE_PYOBJ_MAP, etype_from_pyobj, Relation, Exists, Not
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    33
from yams import BASE_TYPES
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    34
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
    35
from cubicweb import ValidationError, Unauthorized, UnknownEid
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    36
from cubicweb.rqlrewrite import RQLRelationRewriter
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8697
diff changeset
    37
from cubicweb import Binary, server
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    38
from cubicweb.rset import ResultSet
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    39
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    40
from cubicweb.utils import QueryCache, RepeatList
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    41
from cubicweb.server.rqlannotation import SQLGenAnnotator, set_qdata
7118
e094b3d4eb95 [server] move EditedEntity class to its own module, to avoid cyclic dependency when needed from e.g. session.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7083
diff changeset
    42
from cubicweb.server.ssplanner import READ_ONLY_RTYPES, add_types_restriction
e094b3d4eb95 [server] move EditedEntity class to its own module, to avoid cyclic dependency when needed from e.g. session.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7083
diff changeset
    43
from cubicweb.server.edition import EditedEntity
9448
3e7cad3967c5 [multi-sources-removal] Drop the "true" multi-sources planner
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9286
diff changeset
    44
from cubicweb.server.ssplanner import SSPlanner
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents: 10366
diff changeset
    45
from cubicweb.statsd_logger import statsd_timeit, statsd_c
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    46
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    47
ETYPE_PYOBJ_MAP[Binary] = 'Bytes'
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    48
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    49
4722
9c13d5db03d9 pylint suggested refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
    50
def empty_rset(rql, args, rqlst=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    51
    """build an empty result set object"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
    return ResultSet([], rql, args, rqlst=rqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    53
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    54
def update_varmap(varmap, selected, table):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    55
    """return a sql schema to store RQL query result"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    56
    for i, term in enumerate(selected):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    57
        key = term.as_string()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    58
        value = '%s.C%s' % (table, i)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    59
        if varmap.get(key, value) != value:
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
    60
            raise Exception('variable name conflict on %s: got %s / %s'
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
    61
                            % (key, value, varmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    62
        varmap[key] = value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    64
# permission utilities ########################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    65
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    66
def check_no_password_selected(rqlst):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    67
    """check that Password entities are not selected"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    68
    for solution in rqlst.solutions:
8863
98c107bd6cd0 [querier] improve 'Password selection not allowed' error message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8697
diff changeset
    69
        for var, etype in solution.iteritems():
98c107bd6cd0 [querier] improve 'Password selection not allowed' error message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8697
diff changeset
    70
            if etype == 'Password':
98c107bd6cd0 [querier] improve 'Password selection not allowed' error message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8697
diff changeset
    71
                raise Unauthorized('Password selection is not allowed (%s)' % var)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    72
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
    73
def term_etype(cnx, term, solution, args):
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    74
    """return the entity type for the given term (a VariableRef or a Constant
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    75
    node)
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    76
    """
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    77
    try:
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    78
        return solution[term.name]
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    79
    except AttributeError:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
    80
        return cnx.entity_metas(term.eval(args))['type']
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    81
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    82
def check_relations_read_access(cnx, select, args):
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    83
    """Raise :exc:`Unauthorized` if the given user doesn't have credentials to
10555
d8df2c7cff1a [server] typo fix
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    84
    read relations used in the given syntax tree
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    85
    """
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    86
    # use `term_etype` since we've to deal with rewritten constants here,
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    87
    # when used as an external source by another repository.
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    88
    # XXX what about local read security w/ those rewritten constants...
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    89
    # XXX constants can also happen in some queries generated by req.find()
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    90
    DBG = (server.DEBUG & server.DBG_SEC) and 'read' in server._SECURITY_CAPS
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    91
    schema = cnx.repo.schema
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    92
    user = cnx.user
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    93
    if select.where is not None:
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    94
        for rel in select.where.iget_nodes(Relation):
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    95
            for solution in select.solutions:
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    96
                # XXX has_text may have specific perm ?
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    97
                if rel.r_type in READ_ONLY_RTYPES:
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    98
                    continue
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    99
                rschema = schema.rschema(rel.r_type)
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   100
                if rschema.final:
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   101
                    eschema = schema.eschema(term_etype(cnx, rel.children[0],
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   102
                                             solution, args))
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   103
                    rdef = eschema.rdef(rschema)
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   104
                else:
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   105
                    rdef = rschema.rdef(term_etype(cnx, rel.children[0],
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   106
                                                   solution, args),
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   107
                                        term_etype(cnx, rel.children[1].children[0],
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   108
                                                   solution, args))
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   109
                if not user.matching_groups(rdef.get_groups('read')):
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   110
                    if DBG:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   111
                        print('check_read_access: %s %s does not match %s' %
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   112
                              (rdef, user.groups, rdef.get_groups('read')))
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   113
                    # XXX rqlexpr not allowed
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   114
                    raise Unauthorized('read', rel.r_type)
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   115
                if DBG:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   116
                    print('check_read_access: %s %s matches %s' %
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   117
                          (rdef, user.groups, rdef.get_groups('read')))
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   118
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   119
def get_local_checks(cnx, rqlst, solution):
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   120
    """Check that the given user has credentials to access data read by the
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   121
    query and return a dict defining necessary "local checks" (i.e. rql
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   122
    expression in read permission defined in the schema) where no group grants
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   123
    him the permission.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   124
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   125
    Returned dictionary's keys are variable names and values the rql expressions
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   126
    for this variable (with the given solution).
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   127
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   128
    Raise :exc:`Unauthorized` if access is known to be defined, i.e. if there is
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   129
    no matching group and no local permissions.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   130
    """
9286
02a491f6fa92 [querier/security] instrument a bit the querier read security checks
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9267
diff changeset
   131
    DBG = (server.DEBUG & server.DBG_SEC) and 'read' in server._SECURITY_CAPS
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   132
    schema = cnx.repo.schema
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   133
    user = cnx.user
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   134
    localchecks = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
    # iterate on defined_vars and not on solutions to ignore column aliases
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
    for varname in rqlst.defined_vars:
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   137
        eschema = schema.eschema(solution[varname])
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   138
        if eschema.final:
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   139
            continue
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   140
        if not user.matching_groups(eschema.get_groups('read')):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
            erqlexprs = eschema.get_rqlexprs('read')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
            if not erqlexprs:
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   143
                ex = Unauthorized('read', solution[varname])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   144
                ex.var = varname
9286
02a491f6fa92 [querier/security] instrument a bit the querier read security checks
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9267
diff changeset
   145
                if DBG:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   146
                    print('check_read_access: %s %s %s %s' %
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   147
                          (varname, eschema, user.groups, eschema.get_groups('read')))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   148
                raise ex
5888
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   149
            # don't insert security on variable only referenced by 'NOT X relation Y' or
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   150
            # 'NOT EXISTS(X relation Y)'
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   151
            varinfo = rqlst.defined_vars[varname].stinfo
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   152
            if varinfo['selected'] or (
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   153
                len([r for r in varinfo['relations']
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   154
                     if (not schema.rschema(r.r_type).final
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   155
                         and ((isinstance(r.parent, Exists) and r.parent.neged(strict=True))
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   156
                              or isinstance(r.parent, Not)))])
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   157
                !=
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   158
                len(varinfo['relations'])):
5888
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   159
                localchecks[varname] = erqlexprs
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
    return localchecks
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   161
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   162
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   163
# Plans #######################################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   164
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   165
class ExecutionPlan(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   166
    """the execution model of a rql query, composed of querier steps"""
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   167
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   168
    def __init__(self, querier, rqlst, args, cnx):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   169
        # original rql syntax tree
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   170
        self.rqlst = rqlst
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   171
        self.args = args or {}
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   172
        # cnx executing the query
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   173
        self.cnx = cnx
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   174
        # quick reference to the system source
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   175
        self.syssource = cnx.repo.system_source
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   176
        # execution steps
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   177
        self.steps = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   178
        # various resource accesors
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   179
        self.querier = querier
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   180
        self.schema = querier.schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   181
        self.sqlannotate = querier.sqlgen_annotate
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   182
        self.rqlhelper = cnx.vreg.rqlhelper
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   183
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   184
    def annotate_rqlst(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   185
        if not self.rqlst.annotated:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   186
            self.rqlhelper.annotate(self.rqlst)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   187
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   188
    def add_step(self, step):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   189
        """add a step to the plan"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   190
        self.steps.append(step)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   191
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   192
    def sqlexec(self, sql, args=None):
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   193
        return self.syssource.sqlexec(self.cnx, sql, args)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   194
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   195
    def execute(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   196
        """execute a plan and return resulting rows"""
9467
ad66d7b3fd48 [multi-sources-removal] Drop no more used federated ('true') multi-sources related code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9463
diff changeset
   197
        for step in self.steps:
ad66d7b3fd48 [multi-sources-removal] Drop no more used federated ('true') multi-sources related code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9463
diff changeset
   198
            result = step.execute()
ad66d7b3fd48 [multi-sources-removal] Drop no more used federated ('true') multi-sources related code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9463
diff changeset
   199
        # the latest executed step contains the full query result
ad66d7b3fd48 [multi-sources-removal] Drop no more used federated ('true') multi-sources related code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9463
diff changeset
   200
        return result
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   201
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   202
    def preprocess(self, union, security=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   203
        """insert security when necessary then annotate rql st for sql generation
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   204
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   205
        return rqlst to actually execute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   206
        """
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   207
        cached = None
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   208
        if security and self.cnx.read_security:
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   209
            # ensure security is turned of when security is inserted,
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   210
            # else we may loop for ever...
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   211
            if self.cnx.transaction_data.get('security-rqlst-cache'):
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   212
                key = self.cache_key
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   213
            else:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   214
                key = None
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   215
            if key is not None and key in self.cnx.transaction_data:
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   216
                cachedunion, args = self.cnx.transaction_data[key]
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   217
                union.children[:] = []
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   218
                for select in cachedunion.children:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   219
                    union.append(select)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   220
                union.has_text_query = cachedunion.has_text_query
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   221
                args.update(self.args)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   222
                self.args = args
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   223
                cached = True
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   224
            else:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   225
                with self.cnx.security_enabled(read=False):
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   226
                    noinvariant = self._insert_security(union)
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   227
                if key is not None:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   228
                    self.cnx.transaction_data[key] = (union, self.args)
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   229
        else:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   230
            noinvariant = ()
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   231
        if cached is None:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   232
            self.rqlhelper.simplify(union)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   233
            self.sqlannotate(union)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   234
            set_qdata(self.schema.rschema, union, noinvariant)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   235
        if union.has_text_query:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   236
            self.cache_key = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   237
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   238
    def _insert_security(self, union):
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   239
        noinvariant = set()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   240
        for select in union.children[:]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   241
            for subquery in select.with_:
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   242
                self._insert_security(subquery.query)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   243
            localchecks, restricted = self._check_permissions(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   244
            if any(localchecks):
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   245
                self.cnx.rql_rewriter.insert_local_checks(
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   246
                    select, self.args, localchecks, restricted, noinvariant)
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   247
        return noinvariant
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   248
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   249
    def _check_permissions(self, rqlst):
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   250
        """Return a dict defining "local checks", i.e. RQLExpression defined in
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   251
        the schema that should be inserted in the original query, together with
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   252
        a set of variable names which requires some security to be inserted.
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   253
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   254
        Solutions where a variable has a type which the user can't definitly
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   255
        read are removed, else if the user *may* read it (i.e. if an rql
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   256
        expression is defined for the "read" permission of the related type),
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   257
        the local checks dict is updated.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   258
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   259
        The local checks dict has entries for each different local check
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   260
        necessary, with associated solutions as value, a local check being
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   261
        defined by a list of 2-uple (variable name, rql expressions) for each
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   262
        variable which has to be checked. Solutions which don't require local
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   263
        checks will be associated to the empty tuple key.
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   264
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   265
        Note rqlst should not have been simplified at this point.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   266
        """
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   267
        cnx = self.cnx
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   268
        msgs = []
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   269
        # dict(varname: eid), allowing to check rql expression for variables
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   270
        # which have a known eid
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   271
        varkwargs = {}
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   272
        if not cnx.transaction_data.get('security-rqlst-cache'):
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   273
            for var in rqlst.defined_vars.itervalues():
5004
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   274
                if var.stinfo['constnode'] is not None:
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   275
                    eid = var.stinfo['constnode'].eval(self.args)
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8697
diff changeset
   276
                    varkwargs[var.name] = int(eid)
8238
087bb529035c [spelling] fix dictionnary -> dictionary typo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8127
diff changeset
   277
        # dictionary of variables restricted for security reason
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   278
        localchecks = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   279
        restricted_vars = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   280
        newsolutions = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   281
        for solution in rqlst.solutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   282
            try:
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   283
                localcheck = get_local_checks(cnx, rqlst, solution)
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8694
diff changeset
   284
            except Unauthorized as ex:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   285
                msg = 'remove %s from solutions since %s has no %s access to %s'
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   286
                msg %= (solution, cnx.user.login, ex.args[0], ex.args[1])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   287
                msgs.append(msg)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   288
                LOGGER.info(msg)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   289
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
                newsolutions.append(solution)
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   291
                # try to benefit of rqlexpr.check cache for entities which
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   292
                # are specified by eid in query'args
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   293
                for varname, eid in varkwargs.iteritems():
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   294
                    try:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   295
                        rqlexprs = localcheck.pop(varname)
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   296
                    except KeyError:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   297
                        continue
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   298
                    # if entity has been added in the current transaction, the
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   299
                    # user can read it whatever rql expressions are associated
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   300
                    # to its type
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   301
                    if cnx.added_in_transaction(eid):
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   302
                        continue
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   303
                    for rqlexpr in rqlexprs:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   304
                        if rqlexpr.check(cnx, eid):
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   305
                            break
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   306
                    else:
8333
baa3ea5eac2b querier: Improve error message when ERQLExpression fails on some entity (closes #2267081)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8238
diff changeset
   307
                        raise Unauthorized('No read acces on %r with eid %i.' % (var, eid))
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   308
                # mark variables protected by an rql expression
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   309
                restricted_vars.update(localcheck)
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   310
                # turn local check into a dict key
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   311
                localcheck = tuple(sorted(localcheck.iteritems()))
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   312
                localchecks.setdefault(localcheck, []).append(solution)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   313
        # raise Unautorized exception if the user can't access to any solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
        if not newsolutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   315
            raise Unauthorized('\n'.join(msgs))
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   316
        # if there is some message, solutions have been modified and must be
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   317
        # reconsidered by the syntax treee
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5989
diff changeset
   318
        if msgs:
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5989
diff changeset
   319
            rqlst.set_possible_types(newsolutions)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   320
        return localchecks, restricted_vars
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   322
    def finalize(self, select, solutions, insertedvars):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
        rqlst = Union()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
        rqlst.append(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
        for mainvarname, rschema, newvarname in insertedvars:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
            nvartype = str(rschema.objects(solutions[0][mainvarname])[0])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   327
            for sol in solutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
                sol[newvarname] = nvartype
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
        select.clean_solutions(solutions)
6129
fea746b60093 [ms] fix two planner bugs: one occuring query such as X created_by U where X in a external source and U may come from an ldap source. The other being that when we've to merge input maps, we were modifying the same tree/solutions while a copy were needed. Also, ensure we add type restrictions, necessary for pyro source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6128
diff changeset
   330
        add_types_restriction(self.schema, select)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
        self.rqlhelper.annotate(rqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   332
        self.preprocess(rqlst, security=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   333
        return rqlst
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   334
4759
af2e6c377c71 cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   335
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   336
class InsertPlan(ExecutionPlan):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   337
    """an execution model specific to the INSERT rql query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   338
    """
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   339
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   340
    def __init__(self, querier, rqlst, args, cnx):
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   341
        ExecutionPlan.__init__(self, querier, rqlst, args, cnx)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   342
        # save originaly selected variable, we may modify this
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   343
        # dictionary for substitution (query parameters)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   344
        self.selected = rqlst.selection
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   345
        # list of rows of entities definition (ssplanner.EditedEntity)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   346
        self.e_defs = [[]]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   347
        # list of new relation definition (3-uple (from_eid, r_type, to_eid)
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   348
        self.r_defs = set()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
        # indexes to track entity definitions bound to relation definitions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   350
        self._r_subj_index = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   351
        self._r_obj_index = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   352
        self._expanded_r_defs = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   353
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   354
    def add_entity_def(self, edef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   355
        """add an entity definition to build"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   356
        self.e_defs[-1].append(edef)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   357
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   358
    def add_relation_def(self, rdef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   359
        """add an relation definition to build"""
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   360
        self.r_defs.add(rdef)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   361
        if not isinstance(rdef[0], int):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   362
            self._r_subj_index.setdefault(rdef[0], []).append(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   363
        if not isinstance(rdef[2], int):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   364
            self._r_obj_index.setdefault(rdef[2], []).append(rdef)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   365
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   366
    def substitute_entity_def(self, edef, edefs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   367
        """substitute an incomplete entity definition by a list of complete
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   368
        equivalents
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   369
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   370
        e.g. on queries such as ::
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   371
          INSERT Personne X, Societe Y: X nom N, Y nom 'toto', X travaille Y
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   372
          WHERE U login 'admin', U login N
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   373
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   374
        X will be inserted as many times as U exists, and so the X travaille Y
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   375
        relations as to be added as many time as X is inserted
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   376
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   377
        if not edefs or not self.e_defs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   378
            # no result, no entity will be created
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   379
            self.e_defs = ()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   380
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   381
        # first remove the incomplete entity definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   382
        colidx = self.e_defs[0].index(edef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   383
        for i, row in enumerate(self.e_defs[:]):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   384
            self.e_defs[i][colidx] = edefs[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
            samplerow = self.e_defs[i]
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   386
            for edef_ in edefs[1:]:
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   387
                row = [ed.clone() for i, ed in enumerate(samplerow)
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   388
                       if i != colidx]
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   389
                row.insert(colidx, edef_)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   390
                self.e_defs.append(row)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   391
        # now, see if this entity def is referenced as subject in some relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   392
        # definition
8697
574bb05e40a4 [toward py3k] rewrite has_key() (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8696
diff changeset
   393
        if edef in self._r_subj_index:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   394
            for rdef in self._r_subj_index[edef]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   395
                expanded = self._expanded(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   396
                result = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   397
                for exp_rdef in expanded:
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   398
                    for edef_ in edefs:
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   399
                        result.append( (edef_, exp_rdef[1], exp_rdef[2]) )
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   400
                self._expanded_r_defs[rdef] = result
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
        # and finally, see if this entity def is referenced as object in some
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   402
        # relation definition
8697
574bb05e40a4 [toward py3k] rewrite has_key() (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8696
diff changeset
   403
        if edef in self._r_obj_index:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   404
            for rdef in self._r_obj_index[edef]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   405
                expanded = self._expanded(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   406
                result = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   407
                for exp_rdef in expanded:
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   408
                    for edef_ in edefs:
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   409
                        result.append( (exp_rdef[0], exp_rdef[1], edef_) )
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   410
                self._expanded_r_defs[rdef] = result
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   411
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   412
    def _expanded(self, rdef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   413
        """return expanded value for the given relation definition"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   414
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   415
            return self._expanded_r_defs[rdef]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   416
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
            self.r_defs.remove(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   418
            return [rdef]
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   419
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   420
    def relation_defs(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
        """return the list for relation definitions to insert"""
8696
0bb18407c053 [toward py3k] rewrite dict.keys() and dict.values() (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8695
diff changeset
   422
        for rdefs in self._expanded_r_defs.itervalues():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
            for rdef in rdefs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   424
                yield rdef
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   425
        for rdef in self.r_defs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   426
            yield rdef
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   427
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   428
    def insert_entity_defs(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   429
        """return eids of inserted entities in a suitable form for the resulting
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   430
        result set, e.g.:
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   431
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   432
        e.g. on queries such as ::
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   433
          INSERT Personne X, Societe Y: X nom N, Y nom 'toto', X travaille Y
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   434
          WHERE U login 'admin', U login N
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   435
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   436
        if there is two entities matching U, the result set will look like
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   437
        [(eidX1, eidY1), (eidX2, eidY2)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   438
        """
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   439
        cnx = self.cnx
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   440
        repo = cnx.repo
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   441
        results = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   442
        for row in self.e_defs:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   443
            results.append([repo.glob_add_entity(cnx, edef)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   444
                            for edef in row])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   445
        return results
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   446
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   447
    def insert_relation_defs(self):
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   448
        cnx = self.cnx
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   449
        repo = cnx.repo
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   450
        edited_entities = {}
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   451
        relations = {}
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   452
        for subj, rtype, obj in self.relation_defs():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   453
            # if a string is given into args instead of an int, we get it here
10612
84468b90e9c1 [py3k] basestring → six.string_types
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10609
diff changeset
   454
            if isinstance(subj, string_types):
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8697
diff changeset
   455
                subj = int(subj)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   456
            elif not isinstance(subj, (int, long)):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   457
                subj = subj.entity.eid
10612
84468b90e9c1 [py3k] basestring → six.string_types
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10609
diff changeset
   458
            if isinstance(obj, string_types):
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8697
diff changeset
   459
                obj = int(obj)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   460
            elif not isinstance(obj, (int, long)):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   461
                obj = obj.entity.eid
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   462
            if repo.schema.rschema(rtype).inlined:
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   463
                if subj not in edited_entities:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   464
                    entity = cnx.entity_from_eid(subj)
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   465
                    edited = EditedEntity(entity)
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   466
                    edited_entities[subj] = edited
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   467
                else:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   468
                    edited = edited_entities[subj]
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   469
                edited.edited_attribute(rtype, obj)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   470
            else:
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   471
                if rtype in relations:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   472
                    relations[rtype].append((subj, obj))
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   473
                else:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   474
                    relations[rtype] = [(subj, obj)]
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   475
        repo.glob_add_relations(cnx, relations)
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   476
        for edited in edited_entities.itervalues():
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   477
            repo.glob_update_entity(cnx, edited)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   478
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   479
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   480
class QuerierHelper(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   481
    """helper class to execute rql queries, putting all things together"""
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   482
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   483
    def __init__(self, repo, schema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   484
        # system info helper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   485
        self._repo = repo
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2101
diff changeset
   486
        # instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   487
        self.set_schema(schema)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   488
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   489
    def set_schema(self, schema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   490
        self.schema = schema
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   491
        repo = self._repo
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   492
        # rql st and solution cache.
7954
a3d3220669d6 [cache] replace lgc.cache with something more appropriate (closes #1921713)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 7779
diff changeset
   493
        self._rql_cache = QueryCache(repo.config['rql-cache-size'])
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   494
        # rql cache key cache. Don't bother using a Cache instance: we should
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   495
        # have a limited number of queries in there, since there are no entries
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   496
        # in this cache for user queries (which have no args)
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   497
        self._rql_ck_cache = {}
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   498
        # some cache usage stats
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   499
        self.cache_hit, self.cache_miss = 0, 0
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   500
        # rql parsing / analysing helper
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   501
        self.solutions = repo.vreg.solutions
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   502
        rqlhelper = repo.vreg.rqlhelper
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   503
        # set backend on the rql helper, will be used for function checking
9460
a2a0bc984863 [config] cleanup/refactor server sources file values handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9448
diff changeset
   504
        rqlhelper.backend = repo.config.system_source_config['db-driver']
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   505
        self._parse = rqlhelper.parse
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   506
        self._annotate = rqlhelper.annotate
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   507
        # rql planner
9448
3e7cad3967c5 [multi-sources-removal] Drop the "true" multi-sources planner
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9286
diff changeset
   508
        self._planner = SSPlanner(schema, rqlhelper)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   509
        # sql generation annotator
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   510
        self.sqlgen_annotate = SQLGenAnnotator(schema).annotate
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   511
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   512
    def parse(self, rql, annotate=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   513
        """return a rql syntax tree for the given rql"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   514
        try:
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   515
            return self._parse(unicode(rql), annotate=annotate)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   516
        except UnicodeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   517
            raise RQLSyntaxError(rql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   518
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   519
    def plan_factory(self, rqlst, args, cnx):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   520
        """create an execution plan for an INSERT RQL query"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   521
        if rqlst.TYPE == 'insert':
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   522
            return InsertPlan(self, rqlst, args, cnx)
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   523
        return ExecutionPlan(self, rqlst, args, cnx)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   524
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents: 10366
diff changeset
   525
    @statsd_timeit
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   526
    def execute(self, cnx, rql, args=None, build_descr=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   527
        """execute a rql query, return resulting rows and their description in
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   528
        a `ResultSet` object
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   529
9701
46c8d8701240 an unicode string -> a unicode string
Rémi Cardona <remi.cardona@logilab.fr>
parents: 9286
diff changeset
   530
        * `rql` should be a Unicode string or a plain ASCII string
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   531
        * `args` the optional parameters dictionary associated to the query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   532
        * `build_descr` is a boolean flag indicating if the description should
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   533
          be built on select queries (if false, the description will be en empty
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   534
          list)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   535
4654
717310b3d576 docstring improvement
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 4533
diff changeset
   536
        on INSERT queries, there will be one row with the eid of each inserted
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   537
        entity
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   538
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   539
        result for DELETE and SET queries is undefined yet
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   540
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   541
        to maximize the rql parsing/analyzing cache performance, you should
4654
717310b3d576 docstring improvement
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 4533
diff changeset
   542
        always use substitute arguments in queries (i.e. avoid query such as
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   543
        'Any X WHERE X eid 123'!)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   544
        """
2621
1b9d08840a0e R [querier] debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2601
diff changeset
   545
        if server.DEBUG & (server.DBG_RQL | server.DBG_SQL):
2629
0d445c2171e4 R [querier] debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2621
diff changeset
   546
            if server.DEBUG & (server.DBG_MORE | server.DBG_SQL):
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   547
                print('*'*80)
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   548
            print('querier input', repr(rql), repr(args))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   549
        # parse the query and binds variables
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   550
        cachekey = (rql,)
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   551
        try:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   552
            if args:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   553
                # search for named args in query which are eids (hence
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   554
                # influencing query's solutions)
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   555
                eidkeys = self._rql_ck_cache[rql]
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   556
                if eidkeys:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   557
                    # if there are some, we need a better cache key, eg (rql +
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   558
                    # entity type of each eid)
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   559
                    try:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   560
                        cachekey = self._repo.querier_cache_key(cnx, rql,
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   561
                                                                args, eidkeys)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   562
                    except UnknownEid:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   563
                        # we want queries such as "Any X WHERE X eid 9999"
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   564
                        # return an empty result instead of raising UnknownEid
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   565
                        return empty_rset(rql, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   566
            rqlst = self._rql_cache[cachekey]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   567
            self.cache_hit += 1
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents: 10366
diff changeset
   568
            statsd_c('cache_hit')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   569
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   570
            self.cache_miss += 1
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents: 10366
diff changeset
   571
            statsd_c('cache_miss')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   572
            rqlst = self.parse(rql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   573
            try:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   574
                # compute solutions for rqlst and return named args in query
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   575
                # which are eids. Notice that if you may not need `eidkeys`, we
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   576
                # have to compute solutions anyway (kept as annotation on the
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   577
                # tree)
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   578
                eidkeys = self.solutions(cnx, rqlst, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   579
            except UnknownEid:
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   580
                # we want queries such as "Any X WHERE X eid 9999" return an
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   581
                # empty result instead of raising UnknownEid
10246
ef106564b523 [server/querier] stop passing a rqlst to empty_rset
Julien Cristau <julien.cristau@logilab.fr>
parents: 10087
diff changeset
   582
                return empty_rset(rql, args)
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   583
            if args and rql not in self._rql_ck_cache:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   584
                self._rql_ck_cache[rql] = eidkeys
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   585
                if eidkeys:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   586
                    cachekey = self._repo.querier_cache_key(cnx, rql, args,
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   587
                                                            eidkeys)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   588
            self._rql_cache[cachekey] = rqlst
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   589
        if rqlst.TYPE != 'select':
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   590
            if cnx.read_security:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   591
                check_no_password_selected(rqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   592
            cachekey = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   593
        else:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   594
            if cnx.read_security:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   595
                for select in rqlst.children:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   596
                    check_no_password_selected(select)
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   597
                    check_relations_read_access(cnx, select, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   598
            # on select query, always copy the cached rqlst so we don't have to
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   599
            # bother modifying it. This is not necessary on write queries since
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   600
            # a new syntax tree is built from them.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   601
            rqlst = rqlst.copy()
9955
60a9cd1b3a4b [CWEP002] Plug the computed relation rewriter in the querier
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9954
diff changeset
   602
            # Rewrite computed relations
60a9cd1b3a4b [CWEP002] Plug the computed relation rewriter in the querier
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9954
diff changeset
   603
            rewriter = RQLRelationRewriter(cnx)
60a9cd1b3a4b [CWEP002] Plug the computed relation rewriter in the querier
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9954
diff changeset
   604
            rewriter.rewrite(rqlst, args)
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   605
            self._annotate(rqlst)
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   606
            if args:
8342
7a5271182ef0 [rql annotation] fix has_text_query detection (if coming from sub-query and if has_text on a column aliases. Closes #2275322
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8333
diff changeset
   607
                # different SQL generated when some argument is None or not (IS
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   608
                # NULL). This should be considered when computing sql cache key
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   609
                cachekey += tuple(sorted([k for k, v in args.iteritems()
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   610
                                          if v is None]))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   611
        # make an execution plan
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   612
        plan = self.plan_factory(rqlst, args, cnx)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   613
        plan.cache_key = cachekey
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   614
        self._planner.build_plan(plan)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   615
        # execute the plan
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   616
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   617
            results = plan.execute()
6361
843684a50e48 [transaction] to avoid potential db corruption, we should rollback systematically in case of ValidationError
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   618
        except (Unauthorized, ValidationError):
843684a50e48 [transaction] to avoid potential db corruption, we should rollback systematically in case of ValidationError
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   619
            # getting an Unauthorized/ValidationError exception means the
9267
24d9b86dfa54 spelling: rollbacked -> rolled back
Rémi Cardona <remi.cardona@logilab.fr>
parents: 9167
diff changeset
   620
            # transaction must be rolled back
4532
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   621
            #
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   622
            # notes:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7237
diff changeset
   623
            # * we should not reset the connections set here, since we don't want the
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   624
            #   connection to loose it during processing
4532
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   625
            # * don't rollback if we're in the commit process, will be handled
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   626
            #   by the connection
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   627
            if cnx.commit_state is None:
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   628
                cnx.commit_state = 'uncommitable'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   629
            raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   630
        # build a description for the results if necessary
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   631
        descr = ()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   632
        if build_descr:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   633
            if rqlst.TYPE == 'select':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   634
                # sample selection
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   635
                if len(rqlst.children) == 1 and len(rqlst.children[0].solutions) == 1:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   636
                    # easy, all lines are identical
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   637
                    selected = rqlst.children[0].selection
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   638
                    solution = rqlst.children[0].solutions[0]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   639
                    description = _make_description(selected, args, solution)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   640
                    descr = RepeatList(len(results), tuple(description))
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   641
                else:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   642
                    # hard, delegate the work :o)
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   643
                    descr = manual_build_descr(cnx, rqlst, args, results)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   644
            elif rqlst.TYPE == 'insert':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   645
                # on insert plan, some entities may have been auto-casted,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   646
                # so compute description manually even if there is only
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   647
                # one solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   648
                basedescr = [None] * len(plan.selected)
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   649
                todetermine = zip(range(len(plan.selected)), repeat(False))
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   650
                descr = _build_descr(cnx, results, basedescr, todetermine)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   651
            # FIXME: get number of affected entities / relations on non
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   652
            # selection queries ?
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   653
        # return a result set object
10087
ed0b076c119b [rset] kill the rset._rqlst cache
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9990
diff changeset
   654
        return ResultSet(results, rql, args, descr)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   655
7083
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   656
    # these are overridden by set_log_methods below
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   657
    # only defining here to prevent pylint from complaining
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   658
    info = warning = error = critical = exception = debug = lambda msg,*a,**kw: None
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   659
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   660
from logging import getLogger
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   661
from cubicweb import set_log_methods
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   662
LOGGER = getLogger('cubicweb.querier')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   663
set_log_methods(QuerierHelper, LOGGER)
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   664
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   665
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   666
def manual_build_descr(cnx, rqlst, args, result):
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   667
    """build a description for a given result by analysing each row
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   668
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   669
    XXX could probably be done more efficiently during execution of query
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   670
    """
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   671
    # not so easy, looks for variable which changes from one solution
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   672
    # to another
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   673
    unstables = rqlst.get_variable_indices()
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   674
    basedescr = []
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   675
    todetermine = []
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   676
    for i in range(len(rqlst.children[0].selection)):
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   677
        ttype = _selection_idx_type(i, rqlst, args)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   678
        if ttype is None or ttype == 'Any':
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   679
            ttype = None
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   680
            isfinal = True
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   681
        else:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   682
            isfinal = ttype in BASE_TYPES
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   683
        if ttype is None or i in unstables:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   684
            basedescr.append(None)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   685
            todetermine.append( (i, isfinal) )
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   686
        else:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   687
            basedescr.append(ttype)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   688
    if not todetermine:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   689
        return RepeatList(len(result), tuple(basedescr))
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   690
    return _build_descr(cnx, result, basedescr, todetermine)
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   691
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   692
def _build_descr(cnx, result, basedescription, todetermine):
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   693
    description = []
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   694
    entity_metas = cnx.entity_metas
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   695
    todel = []
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   696
    for i, row in enumerate(result):
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   697
        row_descr = basedescription[:]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   698
        for index, isfinal in todetermine:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   699
            value = row[index]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   700
            if value is None:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   701
                # None value inserted by an outer join, no type
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   702
                row_descr[index] = None
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   703
                continue
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   704
            if isfinal:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   705
                row_descr[index] = etype_from_pyobj(value)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   706
            else:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   707
                try:
9469
032825bbacab [multi-sources-removal] Drop entities.source column
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9467
diff changeset
   708
                    row_descr[index] = entity_metas(value)['type']
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   709
                except UnknownEid:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   710
                    cnx.error('wrong eid %s in repository, you should '
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   711
                             'db-check the database' % value)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   712
                    todel.append(i)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   713
                    break
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   714
        else:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   715
            description.append(tuple(row_descr))
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   716
    for i in reversed(todel):
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   717
        del result[i]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   718
    return description
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   719
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   720
def _make_description(selected, args, solution):
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   721
    """return a description for a result set"""
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   722
    description = []
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   723
    for term in selected:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   724
        description.append(term.get_type(solution, args))
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   725
    return description
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   726
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   727
def _selection_idx_type(i, rqlst, args):
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   728
    """try to return type of term at index `i` of the rqlst's selection"""
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   729
    for select in rqlst.children:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   730
        term = select.selection[i]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   731
        for solution in select.solutions:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   732
            try:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   733
                ttype = term.get_type(solution, args)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   734
                if ttype is not None:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   735
                    return ttype
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   736
            except CoercionError:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   737
                return None