server/querier.py
author Aurelien Campeas <aurelien.campeas@logilab.fr>
Fri, 06 Dec 2013 15:56:54 +0100
branchstable
changeset 9338 7f67db7c848e
parent 9267 24d9b86dfa54
child 9286 02a491f6fa92
permissions -rw-r--r--
prepare 3.17.11
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
     1
# copyright 2003-2013 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
"""Helper classes to execute RQL queries on a set of sources, performing
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
security checking and data aggregation.
5726
c3b99606644d [json] fix json serialization for recent simplejson implementation, and test encoding of entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5657
diff changeset
    20
"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
from itertools import repeat
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
from logilab.common.compat import any
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    26
from rql import RQLSyntaxError, CoercionError
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
    27
from rql.stmts import Union
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
    28
from rql.nodes import ETYPE_PYOBJ_MAP, etype_from_pyobj, Relation, Exists, Not
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    29
from yams import BASE_TYPES
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    30
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
    31
from cubicweb import ValidationError, Unauthorized, UnknownEid
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8697
diff changeset
    32
from cubicweb import Binary, server
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    33
from cubicweb.rset import ResultSet
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    34
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    35
from cubicweb.utils import QueryCache, RepeatList
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    36
from cubicweb.server.rqlannotation import SQLGenAnnotator, set_qdata
7118
e094b3d4eb95 [server] move EditedEntity class to its own module, to avoid cyclic dependency when needed from e.g. session.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7083
diff changeset
    37
from cubicweb.server.ssplanner import READ_ONLY_RTYPES, add_types_restriction
e094b3d4eb95 [server] move EditedEntity class to its own module, to avoid cyclic dependency when needed from e.g. session.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7083
diff changeset
    38
from cubicweb.server.edition import EditedEntity
2601
5196d213f4e3 [R querier] new READ_ONLY_RTYPES constant
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2593
diff changeset
    39
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    40
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    41
ETYPE_PYOBJ_MAP[Binary] = 'Bytes'
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    42
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
    43
4722
9c13d5db03d9 pylint suggested refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
    44
def empty_rset(rql, args, rqlst=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    45
    """build an empty result set object"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    46
    return ResultSet([], rql, args, rqlst=rqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    47
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    48
def update_varmap(varmap, selected, table):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    49
    """return a sql schema to store RQL query result"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
    for i, term in enumerate(selected):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    51
        key = term.as_string()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
        value = '%s.C%s' % (table, i)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    53
        if varmap.get(key, value) != value:
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
    54
            raise Exception('variable name conflict on %s: got %s / %s'
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
    55
                            % (key, value, varmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    56
        varmap[key] = value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    57
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    58
# permission utilities ########################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    59
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    60
def check_no_password_selected(rqlst):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    61
    """check that Password entities are not selected"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    62
    for solution in rqlst.solutions:
8863
98c107bd6cd0 [querier] improve 'Password selection not allowed' error message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8697
diff changeset
    63
        for var, etype in solution.iteritems():
98c107bd6cd0 [querier] improve 'Password selection not allowed' error message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8697
diff changeset
    64
            if etype == 'Password':
98c107bd6cd0 [querier] improve 'Password selection not allowed' error message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8697
diff changeset
    65
                raise Unauthorized('Password selection is not allowed (%s)' % var)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    66
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    67
def term_etype(session, term, solution, args):
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    68
    """return the entity type for the given term (a VariableRef or a Constant
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    69
    node)
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    70
    """
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    71
    try:
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    72
        return solution[term.name]
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    73
    except AttributeError:
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    74
        return session.describe(term.eval(args))[0]
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    75
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    76
def check_read_access(session, rqlst, solution, args):
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
    77
    """Check that the given user has credentials to access data read by the
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
    78
    query and return a dict defining necessary "local checks" (i.e. rql
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
    79
    expression in read permission defined in the schema) where no group grants
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
    80
    him the permission.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    81
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
    82
    Returned dictionary's keys are variable names and values the rql expressions
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
    83
    for this variable (with the given solution).
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    84
    """
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    85
    # use `term_etype` since we've to deal with rewritten constants here,
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    86
    # when used as an external source by another repository.
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    87
    # XXX what about local read security w/ those rewritten constants...
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    88
    schema = session.repo.schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    89
    if rqlst.where is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
        for rel in rqlst.where.iget_nodes(Relation):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
            # XXX has_text may have specific perm ?
2601
5196d213f4e3 [R querier] new READ_ONLY_RTYPES constant
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2593
diff changeset
    92
            if rel.r_type in READ_ONLY_RTYPES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
                continue
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    94
            rschema = schema.rschema(rel.r_type)
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    95
            if rschema.final:
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    96
                eschema = schema.eschema(term_etype(session, rel.children[0],
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    97
                                                    solution, args))
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    98
                rdef = eschema.rdef(rschema)
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    99
            else:
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   100
                rdef = rschema.rdef(term_etype(session, rel.children[0],
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   101
                                               solution, args),
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   102
                                    term_etype(session, rel.children[1].children[0],
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   103
                                               solution, args))
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   104
            if not session.user.matching_groups(rdef.get_groups('read')):
4955
8ddd5e938804 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4954
diff changeset
   105
                # XXX rqlexpr not allowed
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   106
                raise Unauthorized('read', rel.r_type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   107
    localchecks = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   108
    # iterate on defined_vars and not on solutions to ignore column aliases
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   109
    for varname in rqlst.defined_vars:
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   110
        eschema = schema.eschema(solution[varname])
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   111
        if eschema.final:
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   112
            continue
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   113
        if not session.user.matching_groups(eschema.get_groups('read')):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   114
            erqlexprs = eschema.get_rqlexprs('read')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   115
            if not erqlexprs:
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   116
                ex = Unauthorized('read', solution[varname])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   117
                ex.var = varname
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   118
                raise ex
5888
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   119
            # don't insert security on variable only referenced by 'NOT X relation Y' or
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   120
            # 'NOT EXISTS(X relation Y)'
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   121
            varinfo = rqlst.defined_vars[varname].stinfo
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   122
            if varinfo['selected'] or (
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   123
                len([r for r in varinfo['relations']
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   124
                     if (not schema.rschema(r.r_type).final
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   125
                         and ((isinstance(r.parent, Exists) and r.parent.neged(strict=True))
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   126
                              or isinstance(r.parent, Not)))])
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   127
                != len(varinfo['relations'])):
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   128
                localchecks[varname] = erqlexprs
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
    return localchecks
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   130
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   131
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   132
# Plans #######################################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   133
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   134
class ExecutionPlan(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
    """the execution model of a rql query, composed of querier steps"""
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   136
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
    def __init__(self, querier, rqlst, args, session):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   138
        # original rql syntax tree
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   139
        self.rqlst = rqlst
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   140
        self.args = args or {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
        # session executing the query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
        self.session = session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   143
        # quick reference to the system source
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7237
diff changeset
   144
        self.syssource = session.cnxset.source('system')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   145
        # execution steps
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   146
        self.steps = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   147
        # index of temporary tables created during execution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   148
        self.temp_tables = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   149
        # various resource accesors
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   150
        self.querier = querier
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   151
        self.schema = querier.schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   152
        self.sqlannotate = querier.sqlgen_annotate
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   153
        self.rqlhelper = session.vreg.rqlhelper
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   154
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   155
    def annotate_rqlst(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   156
        if not self.rqlst.annotated:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   157
            self.rqlhelper.annotate(self.rqlst)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   158
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   159
    def add_step(self, step):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
        """add a step to the plan"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   161
        self.steps.append(step)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   162
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   163
    def clean(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   164
        """remove temporary tables"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   165
        self.syssource.clean_temp_data(self.session, self.temp_tables)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   166
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   167
    def sqlexec(self, sql, args=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   168
        return self.syssource.sqlexec(self.session, sql, args)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   169
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   170
    def execute(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   171
        """execute a plan and return resulting rows"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   172
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   173
            for step in self.steps:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   174
                result = step.execute()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   175
            # the latest executed step contains the full query result
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   176
            return result
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   177
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   178
            self.clean()
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   179
5168
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   180
    def make_temp_table_name(self, table):
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   181
        """
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   182
        return a temp table name according to db backend
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   183
        """
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   184
        return self.syssource.make_temp_table_name(table)
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   185
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   186
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   187
    def init_temp_table(self, table, selected, sol):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   188
        """initialize sql schema and variable map for a temporary table which
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   189
        will be used to store result for the given rqlst
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   190
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   191
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   192
            outputmap, sqlschema, _ = self.temp_tables[table]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   193
            update_varmap(outputmap, selected, table)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   194
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   195
            sqlschema, outputmap = self.syssource.temp_table_def(selected, sol,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   196
                                                                 table)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   197
            self.temp_tables[table] = [outputmap, sqlschema, False]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   198
        return outputmap
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   199
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   200
    def create_temp_table(self, table):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   201
        """create a temporary table to store result for the given rqlst"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   202
        if not self.temp_tables[table][-1]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   203
            sqlschema = self.temp_tables[table][1]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   204
            self.syssource.create_temp_table(self.session, table, sqlschema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   205
            self.temp_tables[table][-1] = True
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   206
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   207
    def preprocess(self, union, security=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   208
        """insert security when necessary then annotate rql st for sql generation
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   209
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   210
        return rqlst to actually execute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   211
        """
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   212
        cached = None
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   213
        if security and self.session.read_security:
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   214
            # ensure security is turned of when security is inserted,
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   215
            # else we may loop for ever...
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   216
            if self.session.transaction_data.get('security-rqlst-cache'):
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   217
                key = self.cache_key
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   218
            else:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   219
                key = None
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   220
            if key is not None and key in self.session.transaction_data:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   221
                cachedunion, args = self.session.transaction_data[key]
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   222
                union.children[:] = []
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   223
                for select in cachedunion.children:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   224
                    union.append(select)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   225
                union.has_text_query = cachedunion.has_text_query
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   226
                args.update(self.args)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   227
                self.args = args
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   228
                cached = True
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   229
            else:
8562
0d2fb4604265 [session] fix arguments default value and promote usage of security_enabled as session method. Closes #2481820
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8542
diff changeset
   230
                with self.session.security_enabled(read=False):
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   231
                    noinvariant = self._insert_security(union)
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   232
                if key is not None:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   233
                    self.session.transaction_data[key] = (union, self.args)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   234
        else:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   235
            noinvariant = ()
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   236
        if cached is None:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   237
            self.rqlhelper.simplify(union)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   238
            self.sqlannotate(union)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   239
            set_qdata(self.schema.rschema, union, noinvariant)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   240
        if union.has_text_query:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   241
            self.cache_key = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   242
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   243
    def _insert_security(self, union):
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   244
        noinvariant = set()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   245
        for select in union.children[:]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   246
            for subquery in select.with_:
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   247
                self._insert_security(subquery.query)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   248
            localchecks, restricted = self._check_permissions(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   249
            if any(localchecks):
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   250
                self.session.rql_rewriter.insert_local_checks(
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   251
                    select, self.args, localchecks, restricted, noinvariant)
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   252
        return noinvariant
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   253
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   254
    def _check_permissions(self, rqlst):
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   255
        """Return a dict defining "local checks", i.e. RQLExpression defined in
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   256
        the schema that should be inserted in the original query, together with
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   257
        a set of variable names which requires some security to be inserted.
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   258
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   259
        Solutions where a variable has a type which the user can't definitly
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   260
        read are removed, else if the user *may* read it (i.e. if an rql
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   261
        expression is defined for the "read" permission of the related type),
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   262
        the local checks dict is updated.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   263
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   264
        The local checks dict has entries for each different local check
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   265
        necessary, with associated solutions as value, a local check being
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   266
        defined by a list of 2-uple (variable name, rql expressions) for each
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   267
        variable which has to be checked. Solutions which don't require local
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   268
        checks will be associated to the empty tuple key.
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   269
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   270
        Note rqlst should not have been simplified at this point.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   271
        """
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   272
        session = self.session
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   273
        msgs = []
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   274
        # dict(varname: eid), allowing to check rql expression for variables
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   275
        # which have a known eid
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   276
        varkwargs = {}
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   277
        if not session.transaction_data.get('security-rqlst-cache'):
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   278
            for var in rqlst.defined_vars.itervalues():
5004
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   279
                if var.stinfo['constnode'] is not None:
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   280
                    eid = var.stinfo['constnode'].eval(self.args)
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8697
diff changeset
   281
                    varkwargs[var.name] = int(eid)
8238
087bb529035c [spelling] fix dictionnary -> dictionary typo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8127
diff changeset
   282
        # dictionary of variables restricted for security reason
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   283
        localchecks = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   284
        restricted_vars = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   285
        newsolutions = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   286
        for solution in rqlst.solutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   287
            try:
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   288
                localcheck = check_read_access(session, rqlst, solution, self.args)
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8694
diff changeset
   289
            except Unauthorized as ex:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
                msg = 'remove %s from solutions since %s has no %s access to %s'
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   291
                msg %= (solution, session.user.login, ex.args[0], ex.args[1])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   292
                msgs.append(msg)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   293
                LOGGER.info(msg)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   294
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   295
                newsolutions.append(solution)
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   296
                # try to benefit of rqlexpr.check cache for entities which
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   297
                # are specified by eid in query'args
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   298
                for varname, eid in varkwargs.iteritems():
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   299
                    try:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   300
                        rqlexprs = localcheck.pop(varname)
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   301
                    except KeyError:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   302
                        continue
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   303
                    # if entity has been added in the current transaction, the
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   304
                    # user can read it whatever rql expressions are associated
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   305
                    # to its type
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   306
                    if session.added_in_transaction(eid):
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   307
                        continue
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   308
                    for rqlexpr in rqlexprs:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   309
                        if rqlexpr.check(session, eid):
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   310
                            break
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   311
                    else:
8333
baa3ea5eac2b querier: Improve error message when ERQLExpression fails on some entity (closes #2267081)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8238
diff changeset
   312
                        raise Unauthorized('No read acces on %r with eid %i.' % (var, eid))
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   313
                # mark variables protected by an rql expression
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
                restricted_vars.update(localcheck)
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   315
                # turn local check into a dict key
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   316
                localcheck = tuple(sorted(localcheck.iteritems()))
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   317
                localchecks.setdefault(localcheck, []).append(solution)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   318
        # raise Unautorized exception if the user can't access to any solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   319
        if not newsolutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   320
            raise Unauthorized('\n'.join(msgs))
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   321
        # if there is some message, solutions have been modified and must be
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   322
        # reconsidered by the syntax treee
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5989
diff changeset
   323
        if msgs:
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5989
diff changeset
   324
            rqlst.set_possible_types(newsolutions)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
        return localchecks, restricted_vars
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   327
    def finalize(self, select, solutions, insertedvars):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
        rqlst = Union()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
        rqlst.append(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   330
        for mainvarname, rschema, newvarname in insertedvars:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
            nvartype = str(rschema.objects(solutions[0][mainvarname])[0])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   332
            for sol in solutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   333
                sol[newvarname] = nvartype
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   334
        select.clean_solutions(solutions)
6129
fea746b60093 [ms] fix two planner bugs: one occuring query such as X created_by U where X in a external source and U may come from an ldap source. The other being that when we've to merge input maps, we were modifying the same tree/solutions while a copy were needed. Also, ensure we add type restrictions, necessary for pyro source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6128
diff changeset
   335
        add_types_restriction(self.schema, select)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   336
        self.rqlhelper.annotate(rqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   337
        self.preprocess(rqlst, security=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   338
        return rqlst
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   339
4759
af2e6c377c71 cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   340
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   341
class InsertPlan(ExecutionPlan):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   342
    """an execution model specific to the INSERT rql query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   343
    """
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   344
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
    def __init__(self, querier, rqlst, args, session):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   346
        ExecutionPlan.__init__(self, querier, rqlst, args, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   347
        # save originaly selected variable, we may modify this
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   348
        # dictionary for substitution (query parameters)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
        self.selected = rqlst.selection
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   350
        # list of rows of entities definition (ssplanner.EditedEntity)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   351
        self.e_defs = [[]]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   352
        # list of new relation definition (3-uple (from_eid, r_type, to_eid)
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   353
        self.r_defs = set()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   354
        # indexes to track entity definitions bound to relation definitions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   355
        self._r_subj_index = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   356
        self._r_obj_index = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   357
        self._expanded_r_defs = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   358
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   359
    def add_entity_def(self, edef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   360
        """add an entity definition to build"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   361
        self.e_defs[-1].append(edef)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   362
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   363
    def add_relation_def(self, rdef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   364
        """add an relation definition to build"""
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   365
        self.r_defs.add(rdef)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   366
        if not isinstance(rdef[0], int):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   367
            self._r_subj_index.setdefault(rdef[0], []).append(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   368
        if not isinstance(rdef[2], int):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   369
            self._r_obj_index.setdefault(rdef[2], []).append(rdef)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   370
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   371
    def substitute_entity_def(self, edef, edefs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   372
        """substitute an incomplete entity definition by a list of complete
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   373
        equivalents
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   374
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   375
        e.g. on queries such as ::
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   376
          INSERT Personne X, Societe Y: X nom N, Y nom 'toto', X travaille Y
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   377
          WHERE U login 'admin', U login N
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   378
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   379
        X will be inserted as many times as U exists, and so the X travaille Y
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   380
        relations as to be added as many time as X is inserted
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   381
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   382
        if not edefs or not self.e_defs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   383
            # no result, no entity will be created
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   384
            self.e_defs = ()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   386
        # first remove the incomplete entity definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   387
        colidx = self.e_defs[0].index(edef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   388
        for i, row in enumerate(self.e_defs[:]):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   389
            self.e_defs[i][colidx] = edefs[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   390
            samplerow = self.e_defs[i]
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   391
            for edef_ in edefs[1:]:
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   392
                row = [ed.clone() for i, ed in enumerate(samplerow)
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   393
                       if i != colidx]
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   394
                row.insert(colidx, edef_)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   395
                self.e_defs.append(row)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   396
        # now, see if this entity def is referenced as subject in some relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   397
        # definition
8697
574bb05e40a4 [toward py3k] rewrite has_key() (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8696
diff changeset
   398
        if edef in self._r_subj_index:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   399
            for rdef in self._r_subj_index[edef]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   400
                expanded = self._expanded(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
                result = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   402
                for exp_rdef in expanded:
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   403
                    for edef_ in edefs:
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   404
                        result.append( (edef_, exp_rdef[1], exp_rdef[2]) )
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   405
                self._expanded_r_defs[rdef] = result
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   406
        # and finally, see if this entity def is referenced as object in some
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   407
        # relation definition
8697
574bb05e40a4 [toward py3k] rewrite has_key() (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8696
diff changeset
   408
        if edef in self._r_obj_index:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   409
            for rdef in self._r_obj_index[edef]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   410
                expanded = self._expanded(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   411
                result = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   412
                for exp_rdef in expanded:
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   413
                    for edef_ in edefs:
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   414
                        result.append( (exp_rdef[0], exp_rdef[1], edef_) )
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   415
                self._expanded_r_defs[rdef] = result
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   416
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
    def _expanded(self, rdef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   418
        """return expanded value for the given relation definition"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   419
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   420
            return self._expanded_r_defs[rdef]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   422
            self.r_defs.remove(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
            return [rdef]
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   424
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   425
    def relation_defs(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   426
        """return the list for relation definitions to insert"""
8696
0bb18407c053 [toward py3k] rewrite dict.keys() and dict.values() (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8695
diff changeset
   427
        for rdefs in self._expanded_r_defs.itervalues():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   428
            for rdef in rdefs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   429
                yield rdef
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   430
        for rdef in self.r_defs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   431
            yield rdef
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   432
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   433
    def insert_entity_defs(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   434
        """return eids of inserted entities in a suitable form for the resulting
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   435
        result set, e.g.:
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   436
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   437
        e.g. on queries such as ::
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   438
          INSERT Personne X, Societe Y: X nom N, Y nom 'toto', X travaille Y
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   439
          WHERE U login 'admin', U login N
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   440
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   441
        if there is two entities matching U, the result set will look like
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   442
        [(eidX1, eidY1), (eidX2, eidY2)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   443
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   444
        session = self.session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   445
        repo = session.repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   446
        results = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   447
        for row in self.e_defs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   448
            results.append([repo.glob_add_entity(session, edef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   449
                            for edef in row])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   450
        return results
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   451
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   452
    def insert_relation_defs(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   453
        session = self.session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   454
        repo = session.repo
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   455
        edited_entities = {}
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   456
        relations = {}
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   457
        for subj, rtype, obj in self.relation_defs():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   458
            # if a string is given into args instead of an int, we get it here
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   459
            if isinstance(subj, basestring):
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8697
diff changeset
   460
                subj = int(subj)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   461
            elif not isinstance(subj, (int, long)):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   462
                subj = subj.entity.eid
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   463
            if isinstance(obj, basestring):
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8697
diff changeset
   464
                obj = int(obj)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   465
            elif not isinstance(obj, (int, long)):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   466
                obj = obj.entity.eid
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   467
            if repo.schema.rschema(rtype).inlined:
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   468
                if subj not in edited_entities:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   469
                    entity = session.entity_from_eid(subj)
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   470
                    edited = EditedEntity(entity)
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   471
                    edited_entities[subj] = edited
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   472
                else:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   473
                    edited = edited_entities[subj]
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   474
                edited.edited_attribute(rtype, obj)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   475
            else:
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   476
                if rtype in relations:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   477
                    relations[rtype].append((subj, obj))
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   478
                else:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   479
                    relations[rtype] = [(subj, obj)]
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   480
        repo.glob_add_relations(session, relations)
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   481
        for edited in edited_entities.itervalues():
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   482
            repo.glob_update_entity(session, edited)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   483
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   484
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   485
class QuerierHelper(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   486
    """helper class to execute rql queries, putting all things together"""
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   487
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   488
    def __init__(self, repo, schema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   489
        # system info helper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   490
        self._repo = repo
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2101
diff changeset
   491
        # instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   492
        self.set_schema(schema)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   493
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   494
    def set_schema(self, schema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   495
        self.schema = schema
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   496
        repo = self._repo
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   497
        # rql st and solution cache.
7954
a3d3220669d6 [cache] replace lgc.cache with something more appropriate (closes #1921713)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 7779
diff changeset
   498
        self._rql_cache = QueryCache(repo.config['rql-cache-size'])
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   499
        # rql cache key cache. Don't bother using a Cache instance: we should
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   500
        # have a limited number of queries in there, since there are no entries
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   501
        # in this cache for user queries (which have no args)
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   502
        self._rql_ck_cache = {}
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   503
        # some cache usage stats
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   504
        self.cache_hit, self.cache_miss = 0, 0
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   505
        # rql parsing / analysing helper
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   506
        self.solutions = repo.vreg.solutions
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   507
        rqlhelper = repo.vreg.rqlhelper
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   508
        # set backend on the rql helper, will be used for function checking
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   509
        rqlhelper.backend = repo.config.sources()['system']['db-driver']
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   510
        self._parse = rqlhelper.parse
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   511
        self._annotate = rqlhelper.annotate
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   512
        # rql planner
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   513
        if len(repo.sources) < 2:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   514
            from cubicweb.server.ssplanner import SSPlanner
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   515
            self._planner = SSPlanner(schema, rqlhelper)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   516
        else:
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   517
            from cubicweb.server.msplanner import MSPlanner
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   518
            self._planner = MSPlanner(schema, rqlhelper)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   519
        # sql generation annotator
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   520
        self.sqlgen_annotate = SQLGenAnnotator(schema).annotate
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   521
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   522
    def set_planner(self):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   523
        if len(self._repo.sources) < 2:
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   524
            from cubicweb.server.ssplanner import SSPlanner
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   525
            self._planner = SSPlanner(self.schema, self._repo.vreg.rqlhelper)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   526
        else:
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   527
            from cubicweb.server.msplanner import MSPlanner
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   528
            self._planner = MSPlanner(self.schema, self._repo.vreg.rqlhelper)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   529
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   530
    def parse(self, rql, annotate=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   531
        """return a rql syntax tree for the given rql"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   532
        try:
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   533
            return self._parse(unicode(rql), annotate=annotate)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   534
        except UnicodeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   535
            raise RQLSyntaxError(rql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   536
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   537
    def plan_factory(self, rqlst, args, session):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   538
        """create an execution plan for an INSERT RQL query"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   539
        if rqlst.TYPE == 'insert':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   540
            return InsertPlan(self, rqlst, args, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   541
        return ExecutionPlan(self, rqlst, args, session)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   542
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   543
    def execute(self, session, rql, args=None, build_descr=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   544
        """execute a rql query, return resulting rows and their description in
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   545
        a `ResultSet` object
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   546
4654
717310b3d576 docstring improvement
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 4533
diff changeset
   547
        * `rql` should be an Unicode string or a plain ASCII string
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   548
        * `args` the optional parameters dictionary associated to the query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   549
        * `build_descr` is a boolean flag indicating if the description should
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   550
          be built on select queries (if false, the description will be en empty
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   551
          list)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   552
4654
717310b3d576 docstring improvement
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 4533
diff changeset
   553
        on INSERT queries, there will be one row with the eid of each inserted
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   554
        entity
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   555
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   556
        result for DELETE and SET queries is undefined yet
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   557
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   558
        to maximize the rql parsing/analyzing cache performance, you should
4654
717310b3d576 docstring improvement
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 4533
diff changeset
   559
        always use substitute arguments in queries (i.e. avoid query such as
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   560
        'Any X WHERE X eid 123'!)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   561
        """
2621
1b9d08840a0e R [querier] debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2601
diff changeset
   562
        if server.DEBUG & (server.DBG_RQL | server.DBG_SQL):
2629
0d445c2171e4 R [querier] debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2621
diff changeset
   563
            if server.DEBUG & (server.DBG_MORE | server.DBG_SQL):
2621
1b9d08840a0e R [querier] debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2601
diff changeset
   564
                print '*'*80
7779
3826d8480a68 [debug] use repr() in some place where you'll have a chance to get an unicode error if output encoding isn't properly detected
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   565
            print 'querier input', repr(rql), repr(args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   566
        # parse the query and binds variables
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   567
        cachekey = (rql,)
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   568
        try:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   569
            if args:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   570
                # search for named args in query which are eids (hence
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   571
                # influencing query's solutions)
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   572
                eidkeys = self._rql_ck_cache[rql]
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   573
                if eidkeys:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   574
                    # if there are some, we need a better cache key, eg (rql +
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   575
                    # entity type of each eid)
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   576
                    try:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   577
                        cachekey = self._repo.querier_cache_key(session, rql,
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   578
                                                                args, eidkeys)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   579
                    except UnknownEid:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   580
                        # we want queries such as "Any X WHERE X eid 9999"
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   581
                        # return an empty result instead of raising UnknownEid
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   582
                        return empty_rset(rql, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   583
            rqlst = self._rql_cache[cachekey]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   584
            self.cache_hit += 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   585
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   586
            self.cache_miss += 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   587
            rqlst = self.parse(rql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   588
            try:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   589
                # compute solutions for rqlst and return named args in query
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   590
                # which are eids. Notice that if you may not need `eidkeys`, we
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   591
                # have to compute solutions anyway (kept as annotation on the
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   592
                # tree)
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   593
                eidkeys = self.solutions(session, rqlst, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   594
            except UnknownEid:
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   595
                # we want queries such as "Any X WHERE X eid 9999" return an
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   596
                # empty result instead of raising UnknownEid
4722
9c13d5db03d9 pylint suggested refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   597
                return empty_rset(rql, args, rqlst)
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   598
            if args and rql not in self._rql_ck_cache:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   599
                self._rql_ck_cache[rql] = eidkeys
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   600
                if eidkeys:
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   601
                    cachekey = self._repo.querier_cache_key(session, rql, args,
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   602
                                                            eidkeys)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   603
            self._rql_cache[cachekey] = rqlst
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   604
        orig_rqlst = rqlst
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   605
        if rqlst.TYPE != 'select':
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   606
            if session.read_security:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   607
                check_no_password_selected(rqlst)
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   608
            # write query, ensure session's mode is 'write' so connections won't
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   609
            # be released until commit/rollback
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   610
            session.mode = 'write'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   611
            cachekey = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   612
        else:
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   613
            if session.read_security:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   614
                for select in rqlst.children:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   615
                    check_no_password_selected(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   616
            # on select query, always copy the cached rqlst so we don't have to
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   617
            # bother modifying it. This is not necessary on write queries since
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   618
            # a new syntax tree is built from them.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   619
            rqlst = rqlst.copy()
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   620
            self._annotate(rqlst)
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   621
            if args:
8342
7a5271182ef0 [rql annotation] fix has_text_query detection (if coming from sub-query and if has_text on a column aliases. Closes #2275322
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8333
diff changeset
   622
                # different SQL generated when some argument is None or not (IS
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   623
                # NULL). This should be considered when computing sql cache key
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8867
diff changeset
   624
                cachekey += tuple(sorted([k for k, v in args.iteritems()
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   625
                                          if v is None]))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   626
        # make an execution plan
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   627
        plan = self.plan_factory(rqlst, args, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   628
        plan.cache_key = cachekey
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   629
        self._planner.build_plan(plan)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   630
        # execute the plan
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   631
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   632
            results = plan.execute()
6361
843684a50e48 [transaction] to avoid potential db corruption, we should rollback systematically in case of ValidationError
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   633
        except (Unauthorized, ValidationError):
843684a50e48 [transaction] to avoid potential db corruption, we should rollback systematically in case of ValidationError
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   634
            # getting an Unauthorized/ValidationError exception means the
9267
24d9b86dfa54 spelling: rollbacked -> rolled back
Rémi Cardona <remi.cardona@logilab.fr>
parents: 9167
diff changeset
   635
            # transaction must be rolled back
4532
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   636
            #
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   637
            # notes:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7237
diff changeset
   638
            # * we should not reset the connections set here, since we don't want the
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7237
diff changeset
   639
            #   session to loose it during processing
4532
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   640
            # * don't rollback if we're in the commit process, will be handled
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   641
            #   by the session
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   642
            if session.commit_state is None:
6385
9f91d09ee5fa [repo transaction] fix rollback behaviour as discussed on the mailing-list: instead of rollbacking automatically on Unauthorized/ValidationError, mark the transaction as uncommitable and disallow commiting
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6361
diff changeset
   643
                session.commit_state = 'uncommitable'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   644
            raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   645
        # build a description for the results if necessary
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   646
        descr = ()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   647
        if build_descr:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   648
            if rqlst.TYPE == 'select':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   649
                # sample selection
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   650
                if len(rqlst.children) == 1 and len(rqlst.children[0].solutions) == 1:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   651
                    # easy, all lines are identical
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   652
                    selected = rqlst.children[0].selection
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   653
                    solution = rqlst.children[0].solutions[0]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   654
                    description = _make_description(selected, args, solution)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   655
                    descr = RepeatList(len(results), tuple(description))
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   656
                else:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   657
                    # hard, delegate the work :o)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   658
                    descr = manual_build_descr(session, rqlst, args, results)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   659
            elif rqlst.TYPE == 'insert':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   660
                # on insert plan, some entities may have been auto-casted,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   661
                # so compute description manually even if there is only
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   662
                # one solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   663
                basedescr = [None] * len(plan.selected)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   664
                todetermine = zip(xrange(len(plan.selected)), repeat(False))
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   665
                descr = _build_descr(session, results, basedescr, todetermine)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   666
            # FIXME: get number of affected entities / relations on non
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   667
            # selection queries ?
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   668
        # return a result set object
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   669
        return ResultSet(results, rql, args, descr, orig_rqlst)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   670
7083
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   671
    # these are overridden by set_log_methods below
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   672
    # only defining here to prevent pylint from complaining
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   673
    info = warning = error = critical = exception = debug = lambda msg,*a,**kw: None
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   674
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   675
from logging import getLogger
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   676
from cubicweb import set_log_methods
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   677
LOGGER = getLogger('cubicweb.querier')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   678
set_log_methods(QuerierHelper, LOGGER)
8542
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   679
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   680
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   681
def manual_build_descr(tx, rqlst, args, result):
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   682
    """build a description for a given result by analysing each row
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   683
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   684
    XXX could probably be done more efficiently during execution of query
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   685
    """
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   686
    # not so easy, looks for variable which changes from one solution
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   687
    # to another
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   688
    unstables = rqlst.get_variable_indices()
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   689
    basedescr = []
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   690
    todetermine = []
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   691
    for i in xrange(len(rqlst.children[0].selection)):
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   692
        ttype = _selection_idx_type(i, rqlst, args)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   693
        if ttype is None or ttype == 'Any':
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   694
            ttype = None
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   695
            isfinal = True
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   696
        else:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   697
            isfinal = ttype in BASE_TYPES
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   698
        if ttype is None or i in unstables:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   699
            basedescr.append(None)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   700
            todetermine.append( (i, isfinal) )
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   701
        else:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   702
            basedescr.append(ttype)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   703
    if not todetermine:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   704
        return RepeatList(len(result), tuple(basedescr))
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   705
    return _build_descr(tx, result, basedescr, todetermine)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   706
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   707
def _build_descr(tx, result, basedescription, todetermine):
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   708
    description = []
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   709
    etype_from_eid = tx.describe
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   710
    todel = []
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   711
    for i, row in enumerate(result):
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   712
        row_descr = basedescription[:]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   713
        for index, isfinal in todetermine:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   714
            value = row[index]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   715
            if value is None:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   716
                # None value inserted by an outer join, no type
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   717
                row_descr[index] = None
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   718
                continue
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   719
            if isfinal:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   720
                row_descr[index] = etype_from_pyobj(value)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   721
            else:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   722
                try:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   723
                    row_descr[index] = etype_from_eid(value)[0]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   724
                except UnknownEid:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   725
                    tx.error('wrong eid %s in repository, you should '
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   726
                             'db-check the database' % value)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   727
                    todel.append(i)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   728
                    break
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   729
        else:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   730
            description.append(tuple(row_descr))
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   731
    for i in reversed(todel):
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   732
        del result[i]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   733
    return description
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   734
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   735
def _make_description(selected, args, solution):
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   736
    """return a description for a result set"""
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   737
    description = []
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   738
    for term in selected:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   739
        description.append(term.get_type(solution, args))
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   740
    return description
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   741
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   742
def _selection_idx_type(i, rqlst, args):
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   743
    """try to return type of term at index `i` of the rqlst's selection"""
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   744
    for select in rqlst.children:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   745
        term = select.selection[i]
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   746
        for solution in select.solutions:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   747
            try:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   748
                ttype = term.get_type(solution, args)
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   749
                if ttype is not None:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   750
                    return ttype
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   751
            except CoercionError:
7e264ce34cd4 [session / querier] reorganize code to building result set descriptions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8342
diff changeset
   752
                return None