server/querier.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Thu, 16 Feb 2012 14:14:52 +0100
branchstable
changeset 8238 087bb529035c
parent 8127 96d343a5e01b
child 8333 baa3ea5eac2b
permissions -rw-r--r--
[spelling] fix dictionnary -> dictionary typo
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
7118
e094b3d4eb95 [server] move EditedEntity class to its own module, to avoid cyclic dependency when needed from e.g. session.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7083
diff changeset
     1
# copyright 2003-2011 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
"""Helper classes to execute RQL queries on a set of sources, performing
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
security checking and data aggregation.
5726
c3b99606644d [json] fix json serialization for recent simplejson implementation, and test encoding of entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5657
diff changeset
    20
"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    22
from __future__ import with_statement
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    23
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
from itertools import repeat
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    28
from logilab.common.compat import any
4719
aaed3f813ef8 kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4654
diff changeset
    29
from rql import RQLSyntaxError
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    30
from rql.stmts import Union, Select
5890
141b935a38fc backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5768 5888
diff changeset
    31
from rql.nodes import (Relation, VariableRef, Constant, SubQuery, Function,
141b935a38fc backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5768 5888
diff changeset
    32
                       Exists, Not)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    33
6361
843684a50e48 [transaction] to avoid potential db corruption, we should rollback systematically in case of ValidationError
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
    34
from cubicweb import ValidationError, Unauthorized, QueryError, UnknownEid
843684a50e48 [transaction] to avoid potential db corruption, we should rollback systematically in case of ValidationError
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
    35
from cubicweb import server, typed_eid
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    36
from cubicweb.rset import ResultSet
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    37
7954
a3d3220669d6 [cache] replace lgc.cache with something more appropriate (closes #1921713)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 7779
diff changeset
    38
from cubicweb.utils import QueryCache
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    39
from cubicweb.server.utils import cleanup_solutions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    40
from cubicweb.server.rqlannotation import SQLGenAnnotator, set_qdata
7118
e094b3d4eb95 [server] move EditedEntity class to its own module, to avoid cyclic dependency when needed from e.g. session.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7083
diff changeset
    41
from cubicweb.server.ssplanner import READ_ONLY_RTYPES, add_types_restriction
e094b3d4eb95 [server] move EditedEntity class to its own module, to avoid cyclic dependency when needed from e.g. session.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7083
diff changeset
    42
from cubicweb.server.edition import EditedEntity
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    43
from cubicweb.server.session import security_enabled
2601
5196d213f4e3 [R querier] new READ_ONLY_RTYPES constant
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2593
diff changeset
    44
4722
9c13d5db03d9 pylint suggested refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
    45
def empty_rset(rql, args, rqlst=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    46
    """build an empty result set object"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    47
    return ResultSet([], rql, args, rqlst=rqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    48
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    49
def update_varmap(varmap, selected, table):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
    """return a sql schema to store RQL query result"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    51
    for i, term in enumerate(selected):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
        key = term.as_string()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    53
        value = '%s.C%s' % (table, i)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    54
        if varmap.get(key, value) != value:
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
    55
            raise Exception('variable name conflict on %s: got %s / %s'
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
    56
                            % (key, value, varmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    57
        varmap[key] = value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    58
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    59
# permission utilities ########################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    60
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    61
def check_no_password_selected(rqlst):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    62
    """check that Password entities are not selected"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
    for solution in rqlst.solutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    64
        if 'Password' in solution.itervalues():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    65
            raise Unauthorized('Password selection is not allowed')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    66
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    67
def term_etype(session, term, solution, args):
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    68
    """return the entity type for the given term (a VariableRef or a Constant
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    69
    node)
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    70
    """
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    71
    try:
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    72
        return solution[term.name]
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    73
    except AttributeError:
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    74
        return session.describe(term.eval(args))[0]
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    75
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    76
def check_read_access(session, rqlst, solution, args):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    77
    """check that the given user has credentials to access data read the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    78
    query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    79
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    80
    return a dict defining necessary local checks (due to use of rql expression
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    81
    in the schema), keys are variable names and values associated rql expression
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    82
    for the associated variable with the given solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    83
    """
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    84
    # use `term_etype` since we've to deal with rewritten constants here,
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    85
    # when used as an external source by another repository.
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    86
    # XXX what about local read security w/ those rewritten constants...
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    87
    schema = session.repo.schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    88
    if rqlst.where is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    89
        for rel in rqlst.where.iget_nodes(Relation):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
            # XXX has_text may have specific perm ?
2601
5196d213f4e3 [R querier] new READ_ONLY_RTYPES constant
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2593
diff changeset
    91
            if rel.r_type in READ_ONLY_RTYPES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
                continue
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    93
            rschema = schema.rschema(rel.r_type)
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    94
            if rschema.final:
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    95
                eschema = schema.eschema(term_etype(session, rel.children[0],
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    96
                                                    solution, args))
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    97
                rdef = eschema.rdef(rschema)
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    98
            else:
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    99
                rdef = rschema.rdef(term_etype(session, rel.children[0],
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   100
                                               solution, args),
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   101
                                    term_etype(session, rel.children[1].children[0],
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   102
                                               solution, args))
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   103
            if not session.user.matching_groups(rdef.get_groups('read')):
4955
8ddd5e938804 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4954
diff changeset
   104
                # XXX rqlexpr not allowed
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   105
                raise Unauthorized('read', rel.r_type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   106
    localchecks = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   107
    # iterate on defined_vars and not on solutions to ignore column aliases
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   108
    for varname in rqlst.defined_vars:
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   109
        eschema = schema.eschema(solution[varname])
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   110
        if eschema.final:
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   111
            continue
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   112
        if not session.user.matching_groups(eschema.get_groups('read')):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   113
            erqlexprs = eschema.get_rqlexprs('read')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   114
            if not erqlexprs:
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   115
                ex = Unauthorized('read', solution[varname])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   116
                ex.var = varname
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   117
                raise ex
5888
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   118
            # don't insert security on variable only referenced by 'NOT X relation Y' or
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   119
            # 'NOT EXISTS(X relation Y)'
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   120
            varinfo = rqlst.defined_vars[varname].stinfo
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   121
            if varinfo['selected'] or (
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   122
                len([r for r in varinfo['relations']
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   123
                     if (not schema.rschema(r.r_type).final
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   124
                         and ((isinstance(r.parent, Exists) and r.parent.neged(strict=True))
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   125
                              or isinstance(r.parent, Not)))])
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   126
                != len(varinfo['relations'])):
3ee80d487f11 [security] fix read rql expression insertion: we should not insert rql expr on variables only referenced in neged relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   127
                localchecks[varname] = erqlexprs
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   128
    return localchecks
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   129
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   130
def add_noinvariant(noinvariant, restricted, select, nbtrees):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   131
    # a variable can actually be invariant if it has not been restricted for
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   132
    # security reason or if security assertion hasn't modified the possible
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   133
    # solutions for the query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   134
    if nbtrees != 1:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
        for vname in restricted:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
            try:
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   137
                noinvariant.add(select.defined_vars[vname])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   138
            except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   139
                # this is an alias
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   140
                continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
        for vname in restricted:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   143
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   144
                var = select.defined_vars[vname]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   145
            except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   146
                # this is an alias
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   147
                continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   148
            if len(var.stinfo['possibletypes']) != 1:
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   149
                noinvariant.add(var)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   150
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   151
def _expand_selection(terms, selected, aliases, select, newselect):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   152
    for term in terms:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   153
        for vref in term.iget_nodes(VariableRef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   154
            if not vref.name in selected:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   155
                select.append_selected(vref)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   156
                colalias = newselect.get_variable(vref.name, len(aliases))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   157
                aliases.append(VariableRef(colalias))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   158
                selected.add(vref.name)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   159
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
# Plans #######################################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   161
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   162
class ExecutionPlan(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   163
    """the execution model of a rql query, composed of querier steps"""
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   164
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   165
    def __init__(self, querier, rqlst, args, session):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   166
        # original rql syntax tree
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   167
        self.rqlst = rqlst
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   168
        self.args = args or {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   169
        # session executing the query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   170
        self.session = session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   171
        # quick reference to the system source
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7237
diff changeset
   172
        self.syssource = session.cnxset.source('system')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   173
        # execution steps
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   174
        self.steps = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   175
        # index of temporary tables created during execution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   176
        self.temp_tables = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   177
        # various resource accesors
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   178
        self.querier = querier
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   179
        self.schema = querier.schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   180
        self.sqlannotate = querier.sqlgen_annotate
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   181
        self.rqlhelper = session.vreg.rqlhelper
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   182
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   183
    def annotate_rqlst(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   184
        if not self.rqlst.annotated:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   185
            self.rqlhelper.annotate(self.rqlst)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   186
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   187
    def add_step(self, step):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   188
        """add a step to the plan"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   189
        self.steps.append(step)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   190
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   191
    def clean(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   192
        """remove temporary tables"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   193
        self.syssource.clean_temp_data(self.session, self.temp_tables)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   194
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   195
    def sqlexec(self, sql, args=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   196
        return self.syssource.sqlexec(self.session, sql, args)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   197
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   198
    def execute(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   199
        """execute a plan and return resulting rows"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   200
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   201
            for step in self.steps:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   202
                result = step.execute()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   203
            # the latest executed step contains the full query result
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   204
            return result
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   205
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   206
            self.clean()
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   207
5168
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   208
    def make_temp_table_name(self, table):
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   209
        """
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   210
        return a temp table name according to db backend
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   211
        """
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   212
        return self.syssource.make_temp_table_name(table)
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   213
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   214
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   215
    def init_temp_table(self, table, selected, sol):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   216
        """initialize sql schema and variable map for a temporary table which
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   217
        will be used to store result for the given rqlst
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   218
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   219
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   220
            outputmap, sqlschema, _ = self.temp_tables[table]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   221
            update_varmap(outputmap, selected, table)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   222
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   223
            sqlschema, outputmap = self.syssource.temp_table_def(selected, sol,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   224
                                                                 table)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   225
            self.temp_tables[table] = [outputmap, sqlschema, False]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   226
        return outputmap
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   227
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   228
    def create_temp_table(self, table):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   229
        """create a temporary table to store result for the given rqlst"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   230
        if not self.temp_tables[table][-1]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   231
            sqlschema = self.temp_tables[table][1]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   232
            self.syssource.create_temp_table(self.session, table, sqlschema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   233
            self.temp_tables[table][-1] = True
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   234
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   235
    def preprocess(self, union, security=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   236
        """insert security when necessary then annotate rql st for sql generation
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   237
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   238
        return rqlst to actually execute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   239
        """
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   240
        cached = None
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   241
        if security and self.session.read_security:
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   242
            # ensure security is turned of when security is inserted,
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   243
            # else we may loop for ever...
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   244
            if self.session.transaction_data.get('security-rqlst-cache'):
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   245
                key = self.cache_key
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   246
            else:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   247
                key = None
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   248
            if key is not None and key in self.session.transaction_data:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   249
                cachedunion, args = self.session.transaction_data[key]
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   250
                union.children[:] = []
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   251
                for select in cachedunion.children:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   252
                    union.append(select)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   253
                union.has_text_query = cachedunion.has_text_query
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   254
                args.update(self.args)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   255
                self.args = args
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   256
                cached = True
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   257
            else:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   258
                noinvariant = set()
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   259
                with security_enabled(self.session, read=False):
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   260
                    self._insert_security(union, noinvariant)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   261
                if key is not None:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   262
                    self.session.transaction_data[key] = (union, self.args)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   263
        else:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   264
            noinvariant = ()
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   265
        if cached is None:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   266
            self.rqlhelper.simplify(union)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   267
            self.sqlannotate(union)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   268
            set_qdata(self.schema.rschema, union, noinvariant)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   269
        if union.has_text_query:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   270
            self.cache_key = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   271
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   272
    def _insert_security(self, union, noinvariant):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   273
        for select in union.children[:]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   274
            for subquery in select.with_:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   275
                self._insert_security(subquery.query, noinvariant)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   276
            localchecks, restricted = self._check_permissions(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   277
            if any(localchecks):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   278
                rewrite = self.session.rql_rewriter.rewrite
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   279
                nbtrees = len(localchecks)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   280
                myunion = union
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   281
                # transform in subquery when len(localchecks)>1 and groups
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   282
                if nbtrees > 1 and (select.orderby or select.groupby or
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   283
                                    select.having or select.has_aggregat or
5596
d66a5d98db5b [security] fix security insertion on DISTINCT queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   284
                                    select.distinct or
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   285
                                    select.limit or select.offset):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   286
                    newselect = Select()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   287
                    # only select variables in subqueries
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   288
                    origselection = select.selection
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   289
                    select.select_only_variables()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
                    select.has_aggregat = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   291
                    # create subquery first so correct node are used on copy
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   292
                    # (eg ColumnAlias instead of Variable)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   293
                    aliases = [VariableRef(newselect.get_variable(vref.name, i))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   294
                               for i, vref in enumerate(select.selection)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   295
                    selected = set(vref.name for vref in aliases)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   296
                    # now copy original selection and groups
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   297
                    for term in origselection:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   298
                        newselect.append_selected(term.copy(newselect))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   299
                    if select.orderby:
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   300
                        sortterms = []
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   301
                        for sortterm in select.orderby:
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   302
                            sortterms.append(sortterm.copy(newselect))
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   303
                            for fnode in sortterm.get_nodes(Function):
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   304
                                if fnode.name == 'FTIRANK':
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   305
                                    # we've to fetch the has_text relation as well
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   306
                                    var = fnode.children[0].variable
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   307
                                    rel = iter(var.stinfo['ftirels']).next()
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   308
                                    assert not rel.ored(), 'unsupported'
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   309
                                    newselect.add_restriction(rel.copy(newselect))
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   310
                                    # remove relation from the orig select and
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   311
                                    # cleanup variable stinfo
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   312
                                    rel.parent.remove(rel)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   313
                                    var.stinfo['ftirels'].remove(rel)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   314
                                    var.stinfo['relations'].remove(rel)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   315
                                    # XXX not properly re-annotated after security insertion?
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   316
                                    newvar = newselect.get_variable(var.name)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   317
                                    newvar.stinfo.setdefault('ftirels', set()).add(rel)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   318
                                    newvar.stinfo.setdefault('relations', set()).add(rel)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   319
                        newselect.set_orderby(sortterms)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   320
                        _expand_selection(select.orderby, selected, aliases, select, newselect)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
                        select.orderby = () # XXX dereference?
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   322
                    if select.groupby:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
                        newselect.set_groupby([g.copy(newselect) for g in select.groupby])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
                        _expand_selection(select.groupby, selected, aliases, select, newselect)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
                        select.groupby = () # XXX dereference?
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
                    if select.having:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   327
                        newselect.set_having([g.copy(newselect) for g in select.having])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
                        _expand_selection(select.having, selected, aliases, select, newselect)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
                        select.having = () # XXX dereference?
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   330
                    if select.limit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
                        newselect.limit = select.limit
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   332
                        select.limit = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   333
                    if select.offset:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   334
                        newselect.offset = select.offset
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   335
                        select.offset = 0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   336
                    myunion = Union()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   337
                    newselect.set_with([SubQuery(aliases, myunion)], check=False)
5596
d66a5d98db5b [security] fix security insertion on DISTINCT queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   338
                    newselect.distinct = select.distinct
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   339
                    solutions = [sol.copy() for sol in select.solutions]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   340
                    cleanup_solutions(newselect, solutions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   341
                    newselect.set_possible_types(solutions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   342
                    # if some solutions doesn't need rewriting, insert original
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   343
                    # select as first union subquery
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   344
                    if () in localchecks:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
                        myunion.append(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   346
                    # we're done, replace original select by the new select with
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   347
                    # subqueries (more added in the loop below)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   348
                    union.replace(select, newselect)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
                elif not () in localchecks:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   350
                    union.remove(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   351
                for lcheckdef, lchecksolutions in localchecks.iteritems():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   352
                    if not lcheckdef:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   353
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   354
                    myrqlst = select.copy(solutions=lchecksolutions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   355
                    myunion.append(myrqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   356
                    # in-place rewrite + annotation / simplification
7139
20807d3d7cf6 [rql rewriter] to properly handle 'relation' rql expressions, rql rewriter must support multiple variables (eg S and O) at once to be given as varmap
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7118
diff changeset
   357
                    lcheckdef = [({var: 'X'}, rqlexprs) for var, rqlexprs in lcheckdef]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   358
                    rewrite(myrqlst, lcheckdef, lchecksolutions, self.args)
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   359
                    add_noinvariant(noinvariant, restricted, myrqlst, nbtrees)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   360
                if () in localchecks:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   361
                    select.set_possible_types(localchecks[()])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   362
                    add_types_restriction(self.schema, select)
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   363
                    add_noinvariant(noinvariant, restricted, select, nbtrees)
5989
099d46422f5a [querier] actually fix bug attempted to be fixed in 5942:f1abba8ccc01. The pb was not in multi-sources but because of missing annotation on rql generated by security insertion
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5890
diff changeset
   364
                self.rqlhelper.annotate(union)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   365
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   366
    def _check_permissions(self, rqlst):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   367
        """return a dict defining "local checks", e.g. RQLExpression defined in
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   368
        the schema that should be inserted in the original query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   369
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   370
        solutions where a variable has a type which the user can't definitly read
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   371
        are removed, else if the user may read it (eg if an rql expression is
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   372
        defined for the "read" permission of the related type), the local checks
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   373
        dict for the solution is updated
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   374
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   375
        return a dict with entries for each different local check necessary,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   376
        with associated solutions as value. A local check is defined by a list
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   377
        of 2-uple, with variable name as first item and the necessary rql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   378
        expression as second item for each variable which has to be checked.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   379
        So solutions which don't require local checks will be associated to
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   380
        the empty tuple key.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   381
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   382
        note: rqlst should not have been simplified at this point
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   383
        """
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   384
        session = self.session
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
        msgs = []
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   386
        neweids = session.transaction_data.get('neweids', ())
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   387
        varkwargs = {}
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   388
        if not session.transaction_data.get('security-rqlst-cache'):
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   389
            for var in rqlst.defined_vars.itervalues():
5004
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   390
                if var.stinfo['constnode'] is not None:
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   391
                    eid = var.stinfo['constnode'].eval(self.args)
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   392
                    varkwargs[var.name] = typed_eid(eid)
8238
087bb529035c [spelling] fix dictionnary -> dictionary typo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8127
diff changeset
   393
        # dictionary of variables restricted for security reason
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   394
        localchecks = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   395
        restricted_vars = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   396
        newsolutions = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   397
        for solution in rqlst.solutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   398
            try:
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   399
                localcheck = check_read_access(session, rqlst, solution, self.args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   400
            except Unauthorized, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
                msg = 'remove %s from solutions since %s has no %s access to %s'
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   402
                msg %= (solution, session.user.login, ex.args[0], ex.args[1])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   403
                msgs.append(msg)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   404
                LOGGER.info(msg)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   405
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   406
                newsolutions.append(solution)
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   407
                # try to benefit of rqlexpr.check cache for entities which
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   408
                # are specified by eid in query'args
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   409
                for varname, eid in varkwargs.iteritems():
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   410
                    try:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   411
                        rqlexprs = localcheck.pop(varname)
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   412
                    except KeyError:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   413
                        continue
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   414
                    if eid in neweids:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   415
                        continue
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   416
                    for rqlexpr in rqlexprs:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   417
                        if rqlexpr.check(session, eid):
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   418
                            break
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   419
                    else:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   420
                        raise Unauthorized()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
                restricted_vars.update(localcheck)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   422
                localchecks.setdefault(tuple(localcheck.iteritems()), []).append(solution)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
        # raise Unautorized exception if the user can't access to any solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   424
        if not newsolutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   425
            raise Unauthorized('\n'.join(msgs))
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5989
diff changeset
   426
        if msgs:
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5989
diff changeset
   427
            # (else solutions have not been modified)
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5989
diff changeset
   428
            rqlst.set_possible_types(newsolutions)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   429
        return localchecks, restricted_vars
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   430
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   431
    def finalize(self, select, solutions, insertedvars):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   432
        rqlst = Union()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   433
        rqlst.append(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   434
        for mainvarname, rschema, newvarname in insertedvars:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   435
            nvartype = str(rschema.objects(solutions[0][mainvarname])[0])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   436
            for sol in solutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   437
                sol[newvarname] = nvartype
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   438
        select.clean_solutions(solutions)
6129
fea746b60093 [ms] fix two planner bugs: one occuring query such as X created_by U where X in a external source and U may come from an ldap source. The other being that when we've to merge input maps, we were modifying the same tree/solutions while a copy were needed. Also, ensure we add type restrictions, necessary for pyro source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6128
diff changeset
   439
        add_types_restriction(self.schema, select)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   440
        self.rqlhelper.annotate(rqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   441
        self.preprocess(rqlst, security=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   442
        return rqlst
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   443
4759
af2e6c377c71 cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   444
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   445
class InsertPlan(ExecutionPlan):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   446
    """an execution model specific to the INSERT rql query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   447
    """
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   448
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   449
    def __init__(self, querier, rqlst, args, session):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   450
        ExecutionPlan.__init__(self, querier, rqlst, args, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   451
        # save originaly selected variable, we may modify this
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   452
        # dictionary for substitution (query parameters)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   453
        self.selected = rqlst.selection
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   454
        # list of rows of entities definition (ssplanner.EditedEntity)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   455
        self.e_defs = [[]]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   456
        # list of new relation definition (3-uple (from_eid, r_type, to_eid)
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   457
        self.r_defs = set()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   458
        # indexes to track entity definitions bound to relation definitions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   459
        self._r_subj_index = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   460
        self._r_obj_index = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   461
        self._expanded_r_defs = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   462
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   463
    def add_entity_def(self, edef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   464
        """add an entity definition to build"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   465
        self.e_defs[-1].append(edef)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   466
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   467
    def add_relation_def(self, rdef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   468
        """add an relation definition to build"""
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   469
        self.r_defs.add(rdef)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   470
        if not isinstance(rdef[0], int):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   471
            self._r_subj_index.setdefault(rdef[0], []).append(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   472
        if not isinstance(rdef[2], int):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   473
            self._r_obj_index.setdefault(rdef[2], []).append(rdef)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   474
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   475
    def substitute_entity_def(self, edef, edefs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   476
        """substitute an incomplete entity definition by a list of complete
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   477
        equivalents
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   478
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   479
        e.g. on queries such as ::
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   480
          INSERT Personne X, Societe Y: X nom N, Y nom 'toto', X travaille Y
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   481
          WHERE U login 'admin', U login N
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   482
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   483
        X will be inserted as many times as U exists, and so the X travaille Y
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   484
        relations as to be added as many time as X is inserted
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   485
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   486
        if not edefs or not self.e_defs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   487
            # no result, no entity will be created
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   488
            self.e_defs = ()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   489
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   490
        # first remove the incomplete entity definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   491
        colidx = self.e_defs[0].index(edef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   492
        for i, row in enumerate(self.e_defs[:]):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   493
            self.e_defs[i][colidx] = edefs[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   494
            samplerow = self.e_defs[i]
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   495
            for edef_ in edefs[1:]:
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   496
                row = [ed.clone() for i, ed in enumerate(samplerow)
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   497
                       if i != colidx]
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   498
                row.insert(colidx, edef_)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   499
                self.e_defs.append(row)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   500
        # now, see if this entity def is referenced as subject in some relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   501
        # definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   502
        if self._r_subj_index.has_key(edef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   503
            for rdef in self._r_subj_index[edef]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   504
                expanded = self._expanded(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   505
                result = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   506
                for exp_rdef in expanded:
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   507
                    for edef_ in edefs:
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   508
                        result.append( (edef_, exp_rdef[1], exp_rdef[2]) )
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   509
                self._expanded_r_defs[rdef] = result
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   510
        # and finally, see if this entity def is referenced as object in some
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   511
        # relation definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   512
        if self._r_obj_index.has_key(edef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   513
            for rdef in self._r_obj_index[edef]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   514
                expanded = self._expanded(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   515
                result = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   516
                for exp_rdef in expanded:
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   517
                    for edef_ in edefs:
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   518
                        result.append( (exp_rdef[0], exp_rdef[1], edef_) )
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   519
                self._expanded_r_defs[rdef] = result
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   520
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   521
    def _expanded(self, rdef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   522
        """return expanded value for the given relation definition"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   523
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   524
            return self._expanded_r_defs[rdef]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   525
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   526
            self.r_defs.remove(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   527
            return [rdef]
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   528
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   529
    def relation_defs(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   530
        """return the list for relation definitions to insert"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   531
        for rdefs in self._expanded_r_defs.values():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   532
            for rdef in rdefs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   533
                yield rdef
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   534
        for rdef in self.r_defs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   535
            yield rdef
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   536
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   537
    def insert_entity_defs(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   538
        """return eids of inserted entities in a suitable form for the resulting
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   539
        result set, e.g.:
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   540
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   541
        e.g. on queries such as ::
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   542
          INSERT Personne X, Societe Y: X nom N, Y nom 'toto', X travaille Y
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   543
          WHERE U login 'admin', U login N
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   544
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   545
        if there is two entities matching U, the result set will look like
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   546
        [(eidX1, eidY1), (eidX2, eidY2)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   547
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   548
        session = self.session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   549
        repo = session.repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   550
        results = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   551
        for row in self.e_defs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   552
            results.append([repo.glob_add_entity(session, edef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   553
                            for edef in row])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   554
        return results
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   555
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   556
    def insert_relation_defs(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   557
        session = self.session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   558
        repo = session.repo
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   559
        edited_entities = {}
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   560
        relations = {}
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   561
        for subj, rtype, obj in self.relation_defs():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   562
            # if a string is given into args instead of an int, we get it here
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   563
            if isinstance(subj, basestring):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   564
                subj = typed_eid(subj)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   565
            elif not isinstance(subj, (int, long)):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   566
                subj = subj.entity.eid
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   567
            if isinstance(obj, basestring):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   568
                obj = typed_eid(obj)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   569
            elif not isinstance(obj, (int, long)):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   570
                obj = obj.entity.eid
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   571
            if repo.schema.rschema(rtype).inlined:
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   572
                if subj not in edited_entities:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   573
                    entity = session.entity_from_eid(subj)
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   574
                    edited = EditedEntity(entity)
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   575
                    edited_entities[subj] = edited
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   576
                else:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   577
                    edited = edited_entities[subj]
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   578
                edited.edited_attribute(rtype, obj)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   579
            else:
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   580
                if rtype in relations:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   581
                    relations[rtype].append((subj, obj))
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   582
                else:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   583
                    relations[rtype] = [(subj, obj)]
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   584
        repo.glob_add_relations(session, relations)
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   585
        for edited in edited_entities.itervalues():
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7139
diff changeset
   586
            repo.glob_update_entity(session, edited)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   587
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   588
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   589
class QuerierHelper(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   590
    """helper class to execute rql queries, putting all things together"""
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   591
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   592
    def __init__(self, repo, schema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   593
        # system info helper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   594
        self._repo = repo
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2101
diff changeset
   595
        # instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   596
        self.set_schema(schema)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   597
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   598
    def set_schema(self, schema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   599
        self.schema = schema
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   600
        repo = self._repo
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   601
        # rql st and solution cache.
7954
a3d3220669d6 [cache] replace lgc.cache with something more appropriate (closes #1921713)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 7779
diff changeset
   602
        self._rql_cache = QueryCache(repo.config['rql-cache-size'])
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   603
        # rql cache key cache. Don't bother using a Cache instance: we should
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   604
        # have a limited number of queries in there, since there are no entries
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   605
        # in this cache for user queries (which have no args)
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   606
        self._rql_ck_cache = {}
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   607
        # some cache usage stats
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   608
        self.cache_hit, self.cache_miss = 0, 0
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   609
        # rql parsing / analysing helper
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   610
        self.solutions = repo.vreg.solutions
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   611
        rqlhelper = repo.vreg.rqlhelper
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   612
        # set backend on the rql helper, will be used for function checking
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   613
        rqlhelper.backend = repo.config.sources()['system']['db-driver']
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   614
        self._parse = rqlhelper.parse
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   615
        self._annotate = rqlhelper.annotate
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   616
        # rql planner
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   617
        if len(repo.sources) < 2:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   618
            from cubicweb.server.ssplanner import SSPlanner
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   619
            self._planner = SSPlanner(schema, rqlhelper)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   620
        else:
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   621
            from cubicweb.server.msplanner import MSPlanner
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   622
            self._planner = MSPlanner(schema, rqlhelper)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   623
        # sql generation annotator
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   624
        self.sqlgen_annotate = SQLGenAnnotator(schema).annotate
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   625
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   626
    def set_planner(self):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   627
        if len(self._repo.sources) < 2:
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   628
            from cubicweb.server.ssplanner import SSPlanner
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   629
            self._planner = SSPlanner(self.schema, self._repo.vreg.rqlhelper)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   630
        else:
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   631
            from cubicweb.server.msplanner import MSPlanner
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   632
            self._planner = MSPlanner(self.schema, self._repo.vreg.rqlhelper)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6401
diff changeset
   633
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   634
    def parse(self, rql, annotate=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   635
        """return a rql syntax tree for the given rql"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   636
        try:
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   637
            return self._parse(unicode(rql), annotate=annotate)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   638
        except UnicodeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   639
            raise RQLSyntaxError(rql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   640
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   641
    def plan_factory(self, rqlst, args, session):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   642
        """create an execution plan for an INSERT RQL query"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   643
        if rqlst.TYPE == 'insert':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   644
            return InsertPlan(self, rqlst, args, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   645
        return ExecutionPlan(self, rqlst, args, session)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   646
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   647
    def execute(self, session, rql, args=None, build_descr=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   648
        """execute a rql query, return resulting rows and their description in
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   649
        a `ResultSet` object
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   650
4654
717310b3d576 docstring improvement
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 4533
diff changeset
   651
        * `rql` should be an Unicode string or a plain ASCII string
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   652
        * `args` the optional parameters dictionary associated to the query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   653
        * `build_descr` is a boolean flag indicating if the description should
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   654
          be built on select queries (if false, the description will be en empty
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   655
          list)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   656
4654
717310b3d576 docstring improvement
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 4533
diff changeset
   657
        on INSERT queries, there will be one row with the eid of each inserted
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   658
        entity
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   659
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   660
        result for DELETE and SET queries is undefined yet
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   661
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   662
        to maximize the rql parsing/analyzing cache performance, you should
4654
717310b3d576 docstring improvement
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 4533
diff changeset
   663
        always use substitute arguments in queries (i.e. avoid query such as
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   664
        'Any X WHERE X eid 123'!)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   665
        """
2621
1b9d08840a0e R [querier] debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2601
diff changeset
   666
        if server.DEBUG & (server.DBG_RQL | server.DBG_SQL):
2629
0d445c2171e4 R [querier] debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2621
diff changeset
   667
            if server.DEBUG & (server.DBG_MORE | server.DBG_SQL):
2621
1b9d08840a0e R [querier] debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2601
diff changeset
   668
                print '*'*80
7779
3826d8480a68 [debug] use repr() in some place where you'll have a chance to get an unicode error if output encoding isn't properly detected
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   669
            print 'querier input', repr(rql), repr(args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   670
        # parse the query and binds variables
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   671
        cachekey = (rql,)
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   672
        try:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   673
            if args:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   674
                # search for named args in query which are eids (hence
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   675
                # influencing query's solutions)
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   676
                eidkeys = self._rql_ck_cache[rql]
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   677
                if eidkeys:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   678
                    # if there are some, we need a better cache key, eg (rql +
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   679
                    # entity type of each eid)
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   680
                    try:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   681
                        cachekey = self._repo.querier_cache_key(session, rql,
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   682
                                                                args, eidkeys)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   683
                    except UnknownEid:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   684
                        # we want queries such as "Any X WHERE X eid 9999"
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   685
                        # return an empty result instead of raising UnknownEid
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   686
                        return empty_rset(rql, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   687
            rqlst = self._rql_cache[cachekey]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   688
            self.cache_hit += 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   689
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   690
            self.cache_miss += 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   691
            rqlst = self.parse(rql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   692
            try:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   693
                # compute solutions for rqlst and return named args in query
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   694
                # which are eids. Notice that if you may not need `eidkeys`, we
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   695
                # have to compute solutions anyway (kept as annotation on the
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   696
                # tree)
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   697
                eidkeys = self.solutions(session, rqlst, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   698
            except UnknownEid:
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   699
                # we want queries such as "Any X WHERE X eid 9999" return an
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   700
                # empty result instead of raising UnknownEid
4722
9c13d5db03d9 pylint suggested refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   701
                return empty_rset(rql, args, rqlst)
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   702
            if args and rql not in self._rql_ck_cache:
7027
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   703
                self._rql_ck_cache[rql] = eidkeys
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   704
                if eidkeys:
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   705
                    cachekey = self._repo.querier_cache_key(session, rql, args,
d3d7ff6ab54c [repo querier] fix rqlst cache no limit issue, comment the code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   706
                                                            eidkeys)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   707
            self._rql_cache[cachekey] = rqlst
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   708
        orig_rqlst = rqlst
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   709
        if rqlst.TYPE != 'select':
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   710
            if session.read_security:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   711
                check_no_password_selected(rqlst)
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   712
            # write query, ensure session's mode is 'write' so connections won't
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   713
            # be released until commit/rollback
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   714
            session.mode = 'write'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   715
            cachekey = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   716
        else:
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   717
            if session.read_security:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   718
                for select in rqlst.children:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   719
                    check_no_password_selected(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   720
            # on select query, always copy the cached rqlst so we don't have to
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   721
            # bother modifying it. This is not necessary on write queries since
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   722
            # a new syntax tree is built from them.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   723
            rqlst = rqlst.copy()
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   724
            self._annotate(rqlst)
8127
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   725
            if args:
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   726
                 # different SQL generated when some argument is None or not (IS
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   727
                # NULL). This should be considered when computing sql cache key
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   728
                cachekey += tuple(sorted([k for k,v in args.iteritems()
96d343a5e01b [rql2sql] None for attributes in kwargs generate IS NULL, so should be considered in sql cache key. Closes #2116693
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7954
diff changeset
   729
                                          if v is None]))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   730
        # make an execution plan
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   731
        plan = self.plan_factory(rqlst, args, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   732
        plan.cache_key = cachekey
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   733
        self._planner.build_plan(plan)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   734
        # execute the plan
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   735
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   736
            results = plan.execute()
6361
843684a50e48 [transaction] to avoid potential db corruption, we should rollback systematically in case of ValidationError
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   737
        except (Unauthorized, ValidationError):
843684a50e48 [transaction] to avoid potential db corruption, we should rollback systematically in case of ValidationError
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   738
            # getting an Unauthorized/ValidationError exception means the
843684a50e48 [transaction] to avoid potential db corruption, we should rollback systematically in case of ValidationError
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6129
diff changeset
   739
            # transaction must been rollbacked
4532
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   740
            #
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   741
            # notes:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7237
diff changeset
   742
            # * we should not reset the connections set here, since we don't want the
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7237
diff changeset
   743
            #   session to loose it during processing
4532
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   744
            # * don't rollback if we're in the commit process, will be handled
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   745
            #   by the session
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   746
            if session.commit_state is None:
6385
9f91d09ee5fa [repo transaction] fix rollback behaviour as discussed on the mailing-list: instead of rollbacking automatically on Unauthorized/ValidationError, mark the transaction as uncommitable and disallow commiting
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6361
diff changeset
   747
                session.commit_state = 'uncommitable'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   748
            raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   749
        # build a description for the results if necessary
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   750
        descr = ()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   751
        if build_descr:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   752
            if rqlst.TYPE == 'select':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   753
                # sample selection
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   754
                descr = session.build_description(orig_rqlst, args, results)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   755
            elif rqlst.TYPE == 'insert':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   756
                # on insert plan, some entities may have been auto-casted,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   757
                # so compute description manually even if there is only
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   758
                # one solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   759
                basedescr = [None] * len(plan.selected)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   760
                todetermine = zip(xrange(len(plan.selected)), repeat(False))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   761
                descr = session._build_descr(results, basedescr, todetermine)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   762
            # FIXME: get number of affected entities / relations on non
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   763
            # selection queries ?
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   764
        # return a result set object
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   765
        return ResultSet(results, rql, args, descr, orig_rqlst)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   766
7083
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   767
    # these are overridden by set_log_methods below
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   768
    # only defining here to prevent pylint from complaining
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   769
    info = warning = error = critical = exception = debug = lambda msg,*a,**kw: None
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7027
diff changeset
   770
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   771
from logging import getLogger
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   772
from cubicweb import set_log_methods
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   773
LOGGER = getLogger('cubicweb.querier')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   774
set_log_methods(QuerierHelper, LOGGER)