server/querier.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Tue, 24 Nov 2015 16:00:06 +0100
changeset 10917 edea9c324396
parent 10903 da30851f9706
permissions -rw-r--r--
[hooks/syncschema] don't set NOT NULL on inlined relation columns The NOT NULL constraint was being added on inlined relation cardinality change while it shouldn't. We don't want this, because this forbids to add an entity without the relation set at once (it may be added later in the transaction). This patch fix synchronization hook to stop doing this. Closes #6211101
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
10087
ed0b076c119b [rset] kill the rset._rqlst cache
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9990
diff changeset
     1
# copyright 2003-2014 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
"""Helper classes to execute RQL queries on a set of sources, performing
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
security checking and data aggregation.
5726
c3b99606644d [json] fix json serialization for recent simplejson implementation, and test encoding of entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5657
diff changeset
    20
"""
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
    21
from __future__ import print_function
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
    22
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
from itertools import repeat
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
10675
e0db937f5add [py3k] unicode → six.text_type
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10663
diff changeset
    27
from six import text_type, string_types, integer_types
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    28
from six.moves import range
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    29
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    30
from rql import RQLSyntaxError, CoercionError
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
    31
from rql.stmts import Union
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
    32
from rql.nodes import ETYPE_PYOBJ_MAP, etype_from_pyobj, Relation, Exists, Not
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    33
from yams import BASE_TYPES
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    34
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
    35
from cubicweb import ValidationError, Unauthorized, UnknownEid
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    36
from cubicweb.rqlrewrite import RQLRelationRewriter
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8697
diff changeset
    37
from cubicweb import Binary, server
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    38
from cubicweb.rset import ResultSet
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    39
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    40
from cubicweb.utils import QueryCache, RepeatList
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    41
from cubicweb.server.rqlannotation import SQLGenAnnotator, set_qdata
7118
e094b3d4eb95 [server] move EditedEntity class to its own module, to avoid cyclic dependency when needed from e.g. session.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7083
diff changeset
    42
from cubicweb.server.ssplanner import READ_ONLY_RTYPES, add_types_restriction
e094b3d4eb95 [server] move EditedEntity class to its own module, to avoid cyclic dependency when needed from e.g. session.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7083
diff changeset
    43
from cubicweb.server.edition import EditedEntity
9448
3e7cad3967c5 [multi-sources-removal] Drop the "true" multi-sources planner
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9286
diff changeset
    44
from cubicweb.server.ssplanner import SSPlanner
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents: 10366
diff changeset
    45
from cubicweb.statsd_logger import statsd_timeit, statsd_c
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    46
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    47
ETYPE_PYOBJ_MAP[Binary] = 'Bytes'
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    48
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    49
4722
9c13d5db03d9 pylint suggested refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
    50
def empty_rset(rql, args, rqlst=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    51
    """build an empty result set object"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
    return ResultSet([], rql, args, rqlst=rqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    53
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    54
def update_varmap(varmap, selected, table):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    55
    """return a sql schema to store RQL query result"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    56
    for i, term in enumerate(selected):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    57
        key = term.as_string()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    58
        value = '%s.C%s' % (table, i)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    59
        if varmap.get(key, value) != value:
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
    60
            raise Exception('variable name conflict on %s: got %s / %s'
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
    61
                            % (key, value, varmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    62
        varmap[key] = value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    64
# permission utilities ########################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    65
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    66
def check_no_password_selected(rqlst):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    67
    """check that Password entities are not selected"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    68
    for solution in rqlst.solutions:
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10613
diff changeset
    69
        for var, etype in solution.items():
8863
98c107bd6cd0 [querier] improve 'Password selection not allowed' error message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8697
diff changeset
    70
            if etype == 'Password':
98c107bd6cd0 [querier] improve 'Password selection not allowed' error message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8697
diff changeset
    71
                raise Unauthorized('Password selection is not allowed (%s)' % var)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    72
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
    73
def term_etype(cnx, term, solution, args):
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    74
    """return the entity type for the given term (a VariableRef or a Constant
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    75
    node)
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    76
    """
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    77
    try:
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    78
        return solution[term.name]
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    79
    except AttributeError:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
    80
        return cnx.entity_metas(term.eval(args))['type']
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    81
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    82
def check_relations_read_access(cnx, select, args):
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    83
    """Raise :exc:`Unauthorized` if the given user doesn't have credentials to
10555
d8df2c7cff1a [server] typo fix
Julien Cristau <julien.cristau@logilab.fr>
parents: 10477
diff changeset
    84
    read relations used in the given syntax tree
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    85
    """
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    86
    # use `term_etype` since we've to deal with rewritten constants here,
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    87
    # when used as an external source by another repository.
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    88
    # XXX what about local read security w/ those rewritten constants...
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    89
    # XXX constants can also happen in some queries generated by req.find()
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    90
    DBG = (server.DEBUG & server.DBG_SEC) and 'read' in server._SECURITY_CAPS
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    91
    schema = cnx.repo.schema
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    92
    user = cnx.user
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    93
    if select.where is not None:
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    94
        for rel in select.where.iget_nodes(Relation):
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    95
            for solution in select.solutions:
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    96
                # XXX has_text may have specific perm ?
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    97
                if rel.r_type in READ_ONLY_RTYPES:
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    98
                    continue
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
    99
                rschema = schema.rschema(rel.r_type)
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   100
                if rschema.final:
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   101
                    eschema = schema.eschema(term_etype(cnx, rel.children[0],
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   102
                                             solution, args))
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   103
                    rdef = eschema.rdef(rschema)
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   104
                else:
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   105
                    rdef = rschema.rdef(term_etype(cnx, rel.children[0],
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   106
                                                   solution, args),
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   107
                                        term_etype(cnx, rel.children[1].children[0],
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   108
                                                   solution, args))
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   109
                if not user.matching_groups(rdef.get_groups('read')):
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   110
                    if DBG:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   111
                        print('check_read_access: %s %s does not match %s' %
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   112
                              (rdef, user.groups, rdef.get_groups('read')))
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   113
                    # XXX rqlexpr not allowed
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   114
                    raise Unauthorized('read', rel.r_type)
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   115
                if DBG:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   116
                    print('check_read_access: %s %s matches %s' %
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   117
                          (rdef, user.groups, rdef.get_groups('read')))
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   118
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   119
def get_local_checks(cnx, rqlst, solution):
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   120
    """Check that the given user has credentials to access data read by the
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   121
    query and return a dict defining necessary "local checks" (i.e. rql
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   122
    expression in read permission defined in the schema) where no group grants
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   123
    him the permission.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   124
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   125
    Returned dictionary's keys are variable names and values the rql expressions
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   126
    for this variable (with the given solution).
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   127
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   128
    Raise :exc:`Unauthorized` if access is known to be defined, i.e. if there is
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   129
    no matching group and no local permissions.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   130
    """
9286
02a491f6fa92 [querier/security] instrument a bit the querier read security checks
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9267
diff changeset
   131
    DBG = (server.DEBUG & server.DBG_SEC) and 'read' in server._SECURITY_CAPS
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   132
    schema = cnx.repo.schema
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   133
    user = cnx.user
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   134
    localchecks = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
    # iterate on defined_vars and not on solutions to ignore column aliases
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
    for varname in rqlst.defined_vars:
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   137
        eschema = schema.eschema(solution[varname])
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   138
        if eschema.final:
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   139
            continue
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   140
        if not user.matching_groups(eschema.get_groups('read')):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
            erqlexprs = eschema.get_rqlexprs('read')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
            if not erqlexprs:
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   143
                ex = Unauthorized('read', solution[varname])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   144
                ex.var = varname
9286
02a491f6fa92 [querier/security] instrument a bit the querier read security checks
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9267
diff changeset
   145
                if DBG:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   146
                    print('check_read_access: %s %s %s %s' %
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   147
                          (varname, eschema, user.groups, eschema.get_groups('read')))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   148
                raise ex
5888
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   149
            # don't insert security on variable only referenced by 'NOT X relation Y' or
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   150
            # 'NOT EXISTS(X relation Y)'
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   151
            varinfo = rqlst.defined_vars[varname].stinfo
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   152
            if varinfo['selected'] or (
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   153
                len([r for r in varinfo['relations']
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   154
                     if (not schema.rschema(r.r_type).final
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   155
                         and ((isinstance(r.parent, Exists) and r.parent.neged(strict=True))
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   156
                              or isinstance(r.parent, Not)))])
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   157
                !=
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   158
                len(varinfo['relations'])):
5888
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   159
                localchecks[varname] = erqlexprs
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
    return localchecks
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   161
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   162
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   163
# Plans #######################################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   164
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   165
class ExecutionPlan(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   166
    """the execution model of a rql query, composed of querier steps"""
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   167
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   168
    def __init__(self, querier, rqlst, args, cnx):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   169
        # original rql syntax tree
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   170
        self.rqlst = rqlst
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   171
        self.args = args or {}
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   172
        # cnx executing the query
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   173
        self.cnx = cnx
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   174
        # quick reference to the system source
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   175
        self.syssource = cnx.repo.system_source
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   176
        # execution steps
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   177
        self.steps = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   178
        # various resource accesors
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   179
        self.querier = querier
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   180
        self.schema = querier.schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   181
        self.sqlannotate = querier.sqlgen_annotate
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   182
        self.rqlhelper = cnx.vreg.rqlhelper
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   183
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   184
    def annotate_rqlst(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   185
        if not self.rqlst.annotated:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   186
            self.rqlhelper.annotate(self.rqlst)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   187
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   188
    def add_step(self, step):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   189
        """add a step to the plan"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   190
        self.steps.append(step)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   191
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   192
    def sqlexec(self, sql, args=None):
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   193
        return self.syssource.sqlexec(self.cnx, sql, args)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   194
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   195
    def execute(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   196
        """execute a plan and return resulting rows"""
9467
ad66d7b3fd48 [multi-sources-removal] Drop no more used federated ('true') multi-sources related code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9463
diff changeset
   197
        for step in self.steps:
ad66d7b3fd48 [multi-sources-removal] Drop no more used federated ('true') multi-sources related code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9463
diff changeset
   198
            result = step.execute()
ad66d7b3fd48 [multi-sources-removal] Drop no more used federated ('true') multi-sources related code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9463
diff changeset
   199
        # the latest executed step contains the full query result
ad66d7b3fd48 [multi-sources-removal] Drop no more used federated ('true') multi-sources related code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9463
diff changeset
   200
        return result
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   201
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   202
    def preprocess(self, union, security=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   203
        """insert security when necessary then annotate rql st for sql generation
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   204
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   205
        return rqlst to actually execute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   206
        """
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   207
        cached = None
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   208
        if security and self.cnx.read_security:
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   209
            # ensure security is turned of when security is inserted,
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   210
            # else we may loop for ever...
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   211
            if self.cnx.transaction_data.get('security-rqlst-cache'):
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   212
                key = self.cache_key
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   213
            else:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   214
                key = None
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   215
            if key is not None and key in self.cnx.transaction_data:
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   216
                cachedunion, args = self.cnx.transaction_data[key]
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   217
                union.children[:] = []
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   218
                for select in cachedunion.children:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   219
                    union.append(select)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   220
                union.has_text_query = cachedunion.has_text_query
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   221
                args.update(self.args)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   222
                self.args = args
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   223
                cached = True
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   224
            else:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   225
                with self.cnx.security_enabled(read=False):
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   226
                    noinvariant = self._insert_security(union)
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   227
                if key is not None:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   228
                    self.cnx.transaction_data[key] = (union, self.args)
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   229
        else:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   230
            noinvariant = ()
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   231
        if cached is None:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   232
            self.rqlhelper.simplify(union)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   233
            self.sqlannotate(union)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   234
            set_qdata(self.schema.rschema, union, noinvariant)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   235
        if union.has_text_query:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   236
            self.cache_key = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   237
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   238
    def _insert_security(self, union):
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   239
        noinvariant = set()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   240
        for select in union.children[:]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   241
            for subquery in select.with_:
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   242
                self._insert_security(subquery.query)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   243
            localchecks, restricted = self._check_permissions(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   244
            if any(localchecks):
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   245
                self.cnx.rql_rewriter.insert_local_checks(
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   246
                    select, self.args, localchecks, restricted, noinvariant)
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   247
        return noinvariant
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   248
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   249
    def _check_permissions(self, rqlst):
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   250
        """Return a dict defining "local checks", i.e. RQLExpression defined in
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   251
        the schema that should be inserted in the original query, together with
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   252
        a set of variable names which requires some security to be inserted.
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   253
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   254
        Solutions where a variable has a type which the user can't definitly
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   255
        read are removed, else if the user *may* read it (i.e. if an rql
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   256
        expression is defined for the "read" permission of the related type),
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   257
        the local checks dict is updated.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   258
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   259
        The local checks dict has entries for each different local check
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   260
        necessary, with associated solutions as value, a local check being
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   261
        defined by a list of 2-uple (variable name, rql expressions) for each
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   262
        variable which has to be checked. Solutions which don't require local
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   263
        checks will be associated to the empty tuple key.
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   264
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   265
        Note rqlst should not have been simplified at this point.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   266
        """
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   267
        cnx = self.cnx
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   268
        msgs = []
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   269
        # dict(varname: eid), allowing to check rql expression for variables
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   270
        # which have a known eid
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   271
        varkwargs = {}
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   272
        if not cnx.transaction_data.get('security-rqlst-cache'):
10663
54b8a1f249fb [py3k] dict.itervalues → dict.values
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10662
diff changeset
   273
            for var in rqlst.defined_vars.values():
5004
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   274
                if var.stinfo['constnode'] is not None:
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   275
                    eid = var.stinfo['constnode'].eval(self.args)
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8697
diff changeset
   276
                    varkwargs[var.name] = int(eid)
8238
087bb529035c [spelling] fix dictionnary -> dictionary typo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8127
diff changeset
   277
        # dictionary of variables restricted for security reason
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   278
        localchecks = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   279
        restricted_vars = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   280
        newsolutions = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   281
        for solution in rqlst.solutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   282
            try:
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   283
                localcheck = get_local_checks(cnx, rqlst, solution)
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8694
diff changeset
   284
            except Unauthorized as ex:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   285
                msg = 'remove %s from solutions since %s has no %s access to %s'
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   286
                msg %= (solution, cnx.user.login, ex.args[0], ex.args[1])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   287
                msgs.append(msg)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   288
                LOGGER.info(msg)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   289
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
                newsolutions.append(solution)
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   291
                # try to benefit of rqlexpr.check cache for entities which
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   292
                # are specified by eid in query'args
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10613
diff changeset
   293
                for varname, eid in varkwargs.items():
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   294
                    try:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   295
                        rqlexprs = localcheck.pop(varname)
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   296
                    except KeyError:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   297
                        continue
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   298
                    # if entity has been added in the current transaction, the
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   299
                    # user can read it whatever rql expressions are associated
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   300
                    # to its type
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   301
                    if cnx.added_in_transaction(eid):
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   302
                        continue
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   303
                    for rqlexpr in rqlexprs:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   304
                        if rqlexpr.check(cnx, eid):
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   305
                            break
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   306
                    else:
8333
baa3ea5eac2b querier: Improve error message when ERQLExpression fails on some entity (closes #2267081)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8238
diff changeset
   307
                        raise Unauthorized('No read acces on %r with eid %i.' % (var, eid))
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   308
                # mark variables protected by an rql expression
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   309
                restricted_vars.update(localcheck)
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   310
                # turn local check into a dict key
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10613
diff changeset
   311
                localcheck = tuple(sorted(localcheck.items()))
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   312
                localchecks.setdefault(localcheck, []).append(solution)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   313
        # raise Unautorized exception if the user can't access to any solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
        if not newsolutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   315
            raise Unauthorized('\n'.join(msgs))
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   316
        # if there is some message, solutions have been modified and must be
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   317
        # reconsidered by the syntax treee
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5989
diff changeset
   318
        if msgs:
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5989
diff changeset
   319
            rqlst.set_possible_types(newsolutions)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   320
        return localchecks, restricted_vars
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   322
    def finalize(self, select, solutions, insertedvars):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
        rqlst = Union()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
        rqlst.append(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
        for mainvarname, rschema, newvarname in insertedvars:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
            nvartype = str(rschema.objects(solutions[0][mainvarname])[0])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   327
            for sol in solutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
                sol[newvarname] = nvartype
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
        select.clean_solutions(solutions)
6129
fea746b60093 [ms] fix two planner bugs: one occuring query such as X created_by U where X in a external source and U may come from an ldap source. The other being that when we've to merge input maps, we were modifying the same tree/solutions while a copy were needed. Also, ensure we add type restrictions, necessary for pyro source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6128
diff changeset
   330
        add_types_restriction(self.schema, select)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
        self.rqlhelper.annotate(rqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   332
        self.preprocess(rqlst, security=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   333
        return rqlst
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   334
4759
af2e6c377c71 cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   335
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   336
class InsertPlan(ExecutionPlan):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   337
    """an execution model specific to the INSERT rql query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   338
    """
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   339
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   340
    def __init__(self, querier, rqlst, args, cnx):
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   341
        ExecutionPlan.__init__(self, querier, rqlst, args, cnx)
10903
da30851f9706 spelling: *aly → *ally
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10759
diff changeset
   342
        # save originally selected variable, we may modify this
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   343
        # dictionary for substitution (query parameters)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   344
        self.selected = rqlst.selection
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   345
        # list of rows of entities definition (ssplanner.EditedEntity)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   346
        self.e_defs = [[]]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   347
        # list of new relation definition (3-uple (from_eid, r_type, to_eid)
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   348
        self.r_defs = set()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
        # indexes to track entity definitions bound to relation definitions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   350
        self._r_subj_index = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   351
        self._r_obj_index = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   352
        self._expanded_r_defs = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   353
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   354
    def add_entity_def(self, edef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   355
        """add an entity definition to build"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   356
        self.e_defs[-1].append(edef)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   357
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   358
    def add_relation_def(self, rdef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   359
        """add an relation definition to build"""
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   360
        self.r_defs.add(rdef)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   361
        if not isinstance(rdef[0], int):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   362
            self._r_subj_index.setdefault(rdef[0], []).append(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   363
        if not isinstance(rdef[2], int):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   364
            self._r_obj_index.setdefault(rdef[2], []).append(rdef)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   365
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   366
    def substitute_entity_def(self, edef, edefs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   367
        """substitute an incomplete entity definition by a list of complete
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   368
        equivalents
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   369
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   370
        e.g. on queries such as ::
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   371
          INSERT Personne X, Societe Y: X nom N, Y nom 'toto', X travaille Y
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   372
          WHERE U login 'admin', U login N
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   373
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   374
        X will be inserted as many times as U exists, and so the X travaille Y
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   375
        relations as to be added as many time as X is inserted
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   376
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   377
        if not edefs or not self.e_defs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   378
            # no result, no entity will be created
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   379
            self.e_defs = ()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   380
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   381
        # first remove the incomplete entity definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   382
        colidx = self.e_defs[0].index(edef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   383
        for i, row in enumerate(self.e_defs[:]):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   384
            self.e_defs[i][colidx] = edefs[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
            samplerow = self.e_defs[i]
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   386
            for edef_ in edefs[1:]:
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   387
                row = [ed.clone() for i, ed in enumerate(samplerow)
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   388
                       if i != colidx]
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   389
                row.insert(colidx, edef_)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   390
                self.e_defs.append(row)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   391
        # now, see if this entity def is referenced as subject in some relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   392
        # definition
8697
574bb05e40a4 [toward py3k] rewrite has_key() (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8696
diff changeset
   393
        if edef in self._r_subj_index:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   394
            for rdef in self._r_subj_index[edef]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   395
                expanded = self._expanded(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   396
                result = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   397
                for exp_rdef in expanded:
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   398
                    for edef_ in edefs:
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   399
                        result.append( (edef_, exp_rdef[1], exp_rdef[2]) )
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   400
                self._expanded_r_defs[rdef] = result
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
        # and finally, see if this entity def is referenced as object in some
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   402
        # relation definition
8697
574bb05e40a4 [toward py3k] rewrite has_key() (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8696
diff changeset
   403
        if edef in self._r_obj_index:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   404
            for rdef in self._r_obj_index[edef]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   405
                expanded = self._expanded(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   406
                result = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   407
                for exp_rdef in expanded:
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   408
                    for edef_ in edefs:
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   409
                        result.append( (exp_rdef[0], exp_rdef[1], edef_) )
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   410
                self._expanded_r_defs[rdef] = result
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   411
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   412
    def _expanded(self, rdef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   413
        """return expanded value for the given relation definition"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   414
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   415
            return self._expanded_r_defs[rdef]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   416
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
            self.r_defs.remove(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   418
            return [rdef]
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   419
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   420
    def relation_defs(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
        """return the list for relation definitions to insert"""
10663
54b8a1f249fb [py3k] dict.itervalues → dict.values
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10662
diff changeset
   422
        for rdefs in self._expanded_r_defs.values():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
            for rdef in rdefs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   424
                yield rdef
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   425
        for rdef in self.r_defs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   426
            yield rdef
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   427
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   428
    def insert_entity_defs(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   429
        """return eids of inserted entities in a suitable form for the resulting
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   430
        result set, e.g.:
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   431
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   432
        e.g. on queries such as ::
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   433
          INSERT Personne X, Societe Y: X nom N, Y nom 'toto', X travaille Y
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   434
          WHERE U login 'admin', U login N
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   435
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   436
        if there is two entities matching U, the result set will look like
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   437
        [(eidX1, eidY1), (eidX2, eidY2)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   438
        """
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   439
        cnx = self.cnx
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   440
        repo = cnx.repo
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   441
        results = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   442
        for row in self.e_defs:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   443
            results.append([repo.glob_add_entity(cnx, edef)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   444
                            for edef in row])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   445
        return results
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   446
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   447
    def insert_relation_defs(self):
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   448
        cnx = self.cnx
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   449
        repo = cnx.repo
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   450
        edited_entities = {}
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   451
        relations = {}
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   452
        for subj, rtype, obj in self.relation_defs():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   453
            # if a string is given into args instead of an int, we get it here
10612
84468b90e9c1 [py3k] basestring → six.string_types
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10609
diff changeset
   454
            if isinstance(subj, string_types):
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8697
diff changeset
   455
                subj = int(subj)
10613
8d9fe02387e3 [py3k] six.integer_types
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10612
diff changeset
   456
            elif not isinstance(subj, integer_types):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   457
                subj = subj.entity.eid
10612
84468b90e9c1 [py3k] basestring → six.string_types
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10609
diff changeset
   458
            if isinstance(obj, string_types):
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8697
diff changeset
   459
                obj = int(obj)
10613
8d9fe02387e3 [py3k] six.integer_types
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10612
diff changeset
   460
            elif not isinstance(obj, integer_types):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   461
                obj = obj.entity.eid
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   462
            if repo.schema.rschema(rtype).inlined:
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   463
                if subj not in edited_entities:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   464
                    entity = cnx.entity_from_eid(subj)
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   465
                    edited = EditedEntity(entity)
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   466
                    edited_entities[subj] = edited
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   467
                else:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   468
                    edited = edited_entities[subj]
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   469
                edited.edited_attribute(rtype, obj)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   470
            else:
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   471
                if rtype in relations:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   472
                    relations[rtype].append((subj, obj))
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   473
                else:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   474
                    relations[rtype] = [(subj, obj)]
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   475
        repo.glob_add_relations(cnx, relations)
10663
54b8a1f249fb [py3k] dict.itervalues → dict.values
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10662
diff changeset
   476
        for edited in edited_entities.values():
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   477
            repo.glob_update_entity(cnx, edited)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   478
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   479
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   480
class QuerierHelper(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   481
    """helper class to execute rql queries, putting all things together"""
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   482
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   483
    def __init__(self, repo, schema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   484
        # system info helper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   485
        self._repo = repo
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2101
diff changeset
   486
        # instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   487
        self.set_schema(schema)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   488
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   489
    def set_schema(self, schema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   490
        self.schema = schema
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   491
        repo = self._repo
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   492
        # rql st and solution cache.
7954
a3d3220669d6 [cache] replace lgc.cache with something more appropriate (closes #1921713)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 7779
diff changeset
   493
        self._rql_cache = QueryCache(repo.config['rql-cache-size'])
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   494
        # rql cache key cache. Don't bother using a Cache instance: we should
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   495
        # have a limited number of queries in there, since there are no entries
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   496
        # in this cache for user queries (which have no args)
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   497
        self._rql_ck_cache = {}
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   498
        # some cache usage stats
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   499
        self.cache_hit, self.cache_miss = 0, 0
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   500
        # rql parsing / analysing helper
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   501
        self.solutions = repo.vreg.solutions
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   502
        rqlhelper = repo.vreg.rqlhelper
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   503
        # set backend on the rql helper, will be used for function checking
9460
a2a0bc984863 [config] cleanup/refactor server sources file values handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9448
diff changeset
   504
        rqlhelper.backend = repo.config.system_source_config['db-driver']
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   505
        self._parse = rqlhelper.parse
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   506
        self._annotate = rqlhelper.annotate
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   507
        # rql planner
9448
3e7cad3967c5 [multi-sources-removal] Drop the "true" multi-sources planner
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9286
diff changeset
   508
        self._planner = SSPlanner(schema, rqlhelper)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   509
        # sql generation annotator
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   510
        self.sqlgen_annotate = SQLGenAnnotator(schema).annotate
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   511
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   512
    def parse(self, rql, annotate=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   513
        """return a rql syntax tree for the given rql"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   514
        try:
10675
e0db937f5add [py3k] unicode → six.text_type
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10663
diff changeset
   515
            return self._parse(text_type(rql), annotate=annotate)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   516
        except UnicodeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   517
            raise RQLSyntaxError(rql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   518
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   519
    def plan_factory(self, rqlst, args, cnx):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   520
        """create an execution plan for an INSERT RQL query"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   521
        if rqlst.TYPE == 'insert':
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   522
            return InsertPlan(self, rqlst, args, cnx)
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   523
        return ExecutionPlan(self, rqlst, args, cnx)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   524
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents: 10366
diff changeset
   525
    @statsd_timeit
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   526
    def execute(self, cnx, rql, args=None, build_descr=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   527
        """execute a rql query, return resulting rows and their description in
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   528
        a `ResultSet` object
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   529
9701
46c8d8701240 an unicode string -> a unicode string
Rémi Cardona <remi.cardona@logilab.fr>
parents: 9286
diff changeset
   530
        * `rql` should be a Unicode string or a plain ASCII string
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   531
        * `args` the optional parameters dictionary associated to the query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   532
        * `build_descr` is a boolean flag indicating if the description should
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   533
          be built on select queries (if false, the description will be en empty
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   534
          list)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   535
4654
717310b3d576 docstring improvement
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 4533
diff changeset
   536
        on INSERT queries, there will be one row with the eid of each inserted
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   537
        entity
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   538
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   539
        result for DELETE and SET queries is undefined yet
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   540
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   541
        to maximize the rql parsing/analyzing cache performance, you should
4654
717310b3d576 docstring improvement
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 4533
diff changeset
   542
        always use substitute arguments in queries (i.e. avoid query such as
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   543
        'Any X WHERE X eid 123'!)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   544
        """
2621
1b9d08840a0e R [querier] debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2601
diff changeset
   545
        if server.DEBUG & (server.DBG_RQL | server.DBG_SQL):
2629
0d445c2171e4 R [querier] debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2621
diff changeset
   546
            if server.DEBUG & (server.DBG_MORE | server.DBG_SQL):
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   547
                print('*'*80)
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10555
diff changeset
   548
            print('querier input', repr(rql), repr(args))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   549
        # parse the query and binds variables
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   550
        cachekey = (rql,)
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   551
        try:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   552
            if args:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   553
                # search for named args in query which are eids (hence
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   554
                # influencing query's solutions)
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   555
                eidkeys = self._rql_ck_cache[rql]
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   556
                if eidkeys:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   557
                    # if there are some, we need a better cache key, eg (rql +
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   558
                    # entity type of each eid)
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   559
                    try:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   560
                        cachekey = self._repo.querier_cache_key(cnx, rql,
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   561
                                                                args, eidkeys)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   562
                    except UnknownEid:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   563
                        # we want queries such as "Any X WHERE X eid 9999"
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   564
                        # return an empty result instead of raising UnknownEid
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   565
                        return empty_rset(rql, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   566
            rqlst = self._rql_cache[cachekey]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   567
            self.cache_hit += 1
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents: 10366
diff changeset
   568
            statsd_c('cache_hit')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   569
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   570
            self.cache_miss += 1
10477
ee21c559f94f implement a simple statsd logger (closes #5488711)
David Douard <david.douard@logilab.fr>
parents: 10366
diff changeset
   571
            statsd_c('cache_miss')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   572
            rqlst = self.parse(rql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   573
            try:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   574
                # compute solutions for rqlst and return named args in query
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   575
                # which are eids. Notice that if you may not need `eidkeys`, we
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   576
                # have to compute solutions anyway (kept as annotation on the
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   577
                # tree)
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   578
                eidkeys = self.solutions(cnx, rqlst, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   579
            except UnknownEid:
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   580
                # we want queries such as "Any X WHERE X eid 9999" return an
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   581
                # empty result instead of raising UnknownEid
10246
ef106564b523 [server/querier] stop passing a rqlst to empty_rset
Julien Cristau <julien.cristau@logilab.fr>
parents: 10087
diff changeset
   582
                return empty_rset(rql, args)
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   583
            if args and rql not in self._rql_ck_cache:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   584
                self._rql_ck_cache[rql] = eidkeys
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   585
                if eidkeys:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   586
                    cachekey = self._repo.querier_cache_key(cnx, rql, args,
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   587
                                                            eidkeys)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   588
            self._rql_cache[cachekey] = rqlst
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   589
        if rqlst.TYPE != 'select':
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   590
            if cnx.read_security:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   591
                check_no_password_selected(rqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   592
            cachekey = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   593
        else:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   594
            if cnx.read_security:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   595
                for select in rqlst.children:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   596
                    check_no_password_selected(select)
9954
79d34ba48612 [CWEP002] refactor rql read security checking
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9892
diff changeset
   597
                    check_relations_read_access(cnx, select, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   598
            # on select query, always copy the cached rqlst so we don't have to
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   599
            # bother modifying it. This is not necessary on write queries since
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   600
            # a new syntax tree is built from them.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   601
            rqlst = rqlst.copy()
9955
60a9cd1b3a4b [CWEP002] Plug the computed relation rewriter in the querier
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9954
diff changeset
   602
            # Rewrite computed relations
60a9cd1b3a4b [CWEP002] Plug the computed relation rewriter in the querier
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9954
diff changeset
   603
            rewriter = RQLRelationRewriter(cnx)
60a9cd1b3a4b [CWEP002] Plug the computed relation rewriter in the querier
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9954
diff changeset
   604
            rewriter.rewrite(rqlst, args)
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   605
            self._annotate(rqlst)
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   606
            if args:
8342
7a5271182ef0 [rql annotation] fix has_text_query detection (if coming from sub-query and if has_text on a column aliases. Closes #2275322
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8333
diff changeset
   607
                # different SQL generated when some argument is None or not (IS
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   608
                # NULL). This should be considered when computing sql cache key
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10613
diff changeset
   609
                cachekey += tuple(sorted([k for k, v in args.items()
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   610
                                          if v is None]))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   611
        # make an execution plan
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   612
        plan = self.plan_factory(rqlst, args, cnx)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   613
        plan.cache_key = cachekey
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   614
        self._planner.build_plan(plan)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   615
        # execute the plan
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   616
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   617
            results = plan.execute()
6361
843684a50e48 [transaction] to avoid potential db corruption, we should rollback systematically in case of ValidationError
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   618
        except (Unauthorized, ValidationError):
843684a50e48 [transaction] to avoid potential db corruption, we should rollback systematically in case of ValidationError
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   619
            # getting an Unauthorized/ValidationError exception means the
9267
24d9b86dfa54 spelling: rollbacked -> rolled back
Rémi Cardona <remi.cardona@logilab.fr>
parents: 9167
diff changeset
   620
            # transaction must be rolled back
4532
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   621
            #
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   622
            # notes:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7237
diff changeset
   623
            # * we should not reset the connections set here, since we don't want the
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   624
            #   connection to loose it during processing
4532
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   625
            # * don't rollback if we're in the commit process, will be handled
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   626
            #   by the connection
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   627
            if cnx.commit_state is None:
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   628
                cnx.commit_state = 'uncommitable'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   629
            raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   630
        # build a description for the results if necessary
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   631
        descr = ()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   632
        if build_descr:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   633
            if rqlst.TYPE == 'select':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   634
                # sample selection
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   635
                if len(rqlst.children) == 1 and len(rqlst.children[0].solutions) == 1:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   636
                    # easy, all lines are identical
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   637
                    selected = rqlst.children[0].selection
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   638
                    solution = rqlst.children[0].solutions[0]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   639
                    description = _make_description(selected, args, solution)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   640
                    descr = RepeatList(len(results), tuple(description))
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   641
                else:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   642
                    # hard, delegate the work :o)
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   643
                    descr = manual_build_descr(cnx, rqlst, args, results)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   644
            elif rqlst.TYPE == 'insert':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   645
                # on insert plan, some entities may have been auto-casted,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   646
                # so compute description manually even if there is only
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   647
                # one solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   648
                basedescr = [None] * len(plan.selected)
10759
774b53f7dc3a [py3k] fix building rset description for INSERT queries
Julien Cristau <julien.cristau@logilab.fr>
parents: 10675
diff changeset
   649
                todetermine = list(zip(range(len(plan.selected)), repeat(False)))
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   650
                descr = _build_descr(cnx, results, basedescr, todetermine)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   651
            # FIXME: get number of affected entities / relations on non
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   652
            # selection queries ?
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   653
        # return a result set object
10087
ed0b076c119b [rset] kill the rset._rqlst cache
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9990
diff changeset
   654
        return ResultSet(results, rql, args, descr)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   655
7083
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   656
    # these are overridden by set_log_methods below
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   657
    # only defining here to prevent pylint from complaining
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   658
    info = warning = error = critical = exception = debug = lambda msg,*a,**kw: None
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   659
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   660
from logging import getLogger
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   661
from cubicweb import set_log_methods
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   662
LOGGER = getLogger('cubicweb.querier')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   663
set_log_methods(QuerierHelper, LOGGER)
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   664
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   665
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   666
def manual_build_descr(cnx, rqlst, args, result):
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   667
    """build a description for a given result by analysing each row
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   668
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   669
    XXX could probably be done more efficiently during execution of query
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   670
    """
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   671
    # not so easy, looks for variable which changes from one solution
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   672
    # to another
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   673
    unstables = rqlst.get_variable_indices()
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   674
    basedescr = []
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   675
    todetermine = []
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   676
    for i in range(len(rqlst.children[0].selection)):
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   677
        ttype = _selection_idx_type(i, rqlst, args)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   678
        if ttype is None or ttype == 'Any':
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   679
            ttype = None
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   680
            isfinal = True
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   681
        else:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   682
            isfinal = ttype in BASE_TYPES
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   683
        if ttype is None or i in unstables:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   684
            basedescr.append(None)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   685
            todetermine.append( (i, isfinal) )
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   686
        else:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   687
            basedescr.append(ttype)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   688
    if not todetermine:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   689
        return RepeatList(len(result), tuple(basedescr))
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   690
    return _build_descr(cnx, result, basedescr, todetermine)
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   691
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   692
def _build_descr(cnx, result, basedescription, todetermine):
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   693
    description = []
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   694
    entity_metas = cnx.entity_metas
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   695
    todel = []
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   696
    for i, row in enumerate(result):
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   697
        row_descr = basedescription[:]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   698
        for index, isfinal in todetermine:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   699
            value = row[index]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   700
            if value is None:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   701
                # None value inserted by an outer join, no type
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   702
                row_descr[index] = None
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   703
                continue
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   704
            if isfinal:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   705
                row_descr[index] = etype_from_pyobj(value)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   706
            else:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   707
                try:
9469
032825bbacab [multi-sources-removal] Drop entities.source column
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9467
diff changeset
   708
                    row_descr[index] = entity_metas(value)['type']
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   709
                except UnknownEid:
9508
1263f1258796 [server] rename session to cnx in querier and plan
Julien Cristau <julien.cristau@logilab.fr>
parents: 9469
diff changeset
   710
                    cnx.error('wrong eid %s in repository, you should '
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   711
                             'db-check the database' % value)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   712
                    todel.append(i)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   713
                    break
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   714
        else:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   715
            description.append(tuple(row_descr))
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   716
    for i in reversed(todel):
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   717
        del result[i]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   718
    return description
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   719
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   720
def _make_description(selected, args, solution):
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   721
    """return a description for a result set"""
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   722
    description = []
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   723
    for term in selected:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   724
        description.append(term.get_type(solution, args))
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   725
    return description
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   726
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   727
def _selection_idx_type(i, rqlst, args):
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   728
    """try to return type of term at index `i` of the rqlst's selection"""
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   729
    for select in rqlst.children:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   730
        term = select.selection[i]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   731
        for solution in select.solutions:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   732
            try:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   733
                ttype = term.get_type(solution, args)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   734
                if ttype is not None:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   735
                    return ttype
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   736
            except CoercionError:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   737
                return None