server/querier.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Thu, 17 Jun 2010 15:15:52 +0200
changeset 5774 0d792bceb25d
parent 5768 1e73a466aa69
child 5890 141b935a38fc
permissions -rw-r--r--
backport stable
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     1
# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5419
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
"""Helper classes to execute RQL queries on a set of sources, performing
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
security checking and data aggregation.
5726
c3b99606644d [json] fix json serialization for recent simplejson implementation, and test encoding of entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5657
diff changeset
    20
"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    22
from __future__ import with_statement
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    23
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
from itertools import repeat
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    28
from logilab.common.cache import Cache
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    29
from logilab.common.compat import any
4719
aaed3f813ef8 kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4654
diff changeset
    30
from rql import RQLSyntaxError
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    31
from rql.stmts import Union, Select
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
    32
from rql.nodes import Relation, VariableRef, Constant, SubQuery, Function
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    33
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    34
from cubicweb import Unauthorized, QueryError, UnknownEid, typed_eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    35
from cubicweb import server
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    36
from cubicweb.rset import ResultSet
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    37
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    38
from cubicweb.server.utils import cleanup_solutions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    39
from cubicweb.server.rqlannotation import SQLGenAnnotator, set_qdata
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4759
diff changeset
    40
from cubicweb.server.ssplanner import READ_ONLY_RTYPES, add_types_restriction
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    41
from cubicweb.server.session import security_enabled
2601
5196d213f4e3 [R querier] new READ_ONLY_RTYPES constant
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2593
diff changeset
    42
4722
9c13d5db03d9 pylint suggested refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
    43
def empty_rset(rql, args, rqlst=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    44
    """build an empty result set object"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    45
    return ResultSet([], rql, args, rqlst=rqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    46
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    47
def update_varmap(varmap, selected, table):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    48
    """return a sql schema to store RQL query result"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    49
    for i, term in enumerate(selected):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
        key = term.as_string()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    51
        value = '%s.C%s' % (table, i)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
        if varmap.get(key, value) != value:
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
    53
            raise Exception('variable name conflict on %s: got %s / %s'
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
    54
                            % (key, value, varmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    55
        varmap[key] = value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    56
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    57
# permission utilities ########################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    58
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    59
def check_no_password_selected(rqlst):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    60
    """check that Password entities are not selected"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    61
    for solution in rqlst.solutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    62
        if 'Password' in solution.itervalues():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
            raise Unauthorized('Password selection is not allowed')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    64
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    65
def term_etype(session, term, solution, args):
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    66
    """return the entity type for the given term (a VariableRef or a Constant
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    67
    node)
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    68
    """
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    69
    try:
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    70
        return solution[term.name]
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    71
    except AttributeError:
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    72
        return session.describe(term.eval(args))[0]
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    73
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    74
def check_read_access(session, rqlst, solution, args):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    75
    """check that the given user has credentials to access data read the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    76
    query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    77
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    78
    return a dict defining necessary local checks (due to use of rql expression
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    79
    in the schema), keys are variable names and values associated rql expression
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    80
    for the associated variable with the given solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    81
    """
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    82
    # use `term_etype` since we've to deal with rewritten constants here,
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    83
    # when used as an external source by another repository.
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    84
    # XXX what about local read security w/ those rewritten constants...
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    85
    schema = session.repo.schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
    if rqlst.where is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
        for rel in rqlst.where.iget_nodes(Relation):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    88
            # XXX has_text may have specific perm ?
2601
5196d213f4e3 [R querier] new READ_ONLY_RTYPES constant
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2593
diff changeset
    89
            if rel.r_type in READ_ONLY_RTYPES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
                continue
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    91
            rschema = schema.rschema(rel.r_type)
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    92
            if rschema.final:
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    93
                eschema = schema.eschema(term_etype(session, rel.children[0],
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    94
                                                    solution, args))
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    95
                rdef = eschema.rdef(rschema)
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    96
            else:
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    97
                rdef = rschema.rdef(term_etype(session, rel.children[0],
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    98
                                               solution, args),
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
    99
                                    term_etype(session, rel.children[1].children[0],
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   100
                                               solution, args))
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   101
            if not session.user.matching_groups(rdef.get_groups('read')):
4955
8ddd5e938804 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4954
diff changeset
   102
                # XXX rqlexpr not allowed
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   103
                raise Unauthorized('read', rel.r_type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   104
    localchecks = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   105
    # iterate on defined_vars and not on solutions to ignore column aliases
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   106
    for varname in rqlst.defined_vars:
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   107
        eschema = schema.eschema(solution[varname])
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   108
        if eschema.final:
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   109
            continue
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   110
        if not session.user.matching_groups(eschema.get_groups('read')):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   111
            erqlexprs = eschema.get_rqlexprs('read')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   112
            if not erqlexprs:
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   113
                ex = Unauthorized('read', solution[varname])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   114
                ex.var = varname
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   115
                raise ex
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   116
            localchecks[varname] = erqlexprs
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   117
    return localchecks
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   118
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   119
def add_noinvariant(noinvariant, restricted, select, nbtrees):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   120
    # a variable can actually be invariant if it has not been restricted for
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   121
    # security reason or if security assertion hasn't modified the possible
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   122
    # solutions for the query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   123
    if nbtrees != 1:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   124
        for vname in restricted:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   125
            try:
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   126
                noinvariant.add(select.defined_vars[vname])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   127
            except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   128
                # this is an alias
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
                continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   130
    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   131
        for vname in restricted:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   132
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   133
                var = select.defined_vars[vname]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   134
            except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
                # this is an alias
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
                continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
            if len(var.stinfo['possibletypes']) != 1:
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   138
                noinvariant.add(var)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   139
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   140
def _expand_selection(terms, selected, aliases, select, newselect):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
    for term in terms:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
        for vref in term.iget_nodes(VariableRef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   143
            if not vref.name in selected:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   144
                select.append_selected(vref)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   145
                colalias = newselect.get_variable(vref.name, len(aliases))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   146
                aliases.append(VariableRef(colalias))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   147
                selected.add(vref.name)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   148
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   149
# Plans #######################################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   150
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   151
class ExecutionPlan(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   152
    """the execution model of a rql query, composed of querier steps"""
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   153
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   154
    def __init__(self, querier, rqlst, args, session):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   155
        # original rql syntax tree
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   156
        self.rqlst = rqlst
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   157
        self.args = args or {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   158
        # session executing the query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   159
        self.session = session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
        # quick reference to the system source
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   161
        self.syssource = session.pool.source('system')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   162
        # execution steps
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   163
        self.steps = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   164
        # index of temporary tables created during execution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   165
        self.temp_tables = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   166
        # various resource accesors
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   167
        self.querier = querier
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   168
        self.schema = querier.schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   169
        self.sqlannotate = querier.sqlgen_annotate
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   170
        self.rqlhelper = session.vreg.rqlhelper
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   171
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   172
    def annotate_rqlst(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   173
        if not self.rqlst.annotated:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   174
            self.rqlhelper.annotate(self.rqlst)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   175
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   176
    def add_step(self, step):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   177
        """add a step to the plan"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   178
        self.steps.append(step)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   179
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   180
    def clean(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   181
        """remove temporary tables"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   182
        self.syssource.clean_temp_data(self.session, self.temp_tables)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   183
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   184
    def sqlexec(self, sql, args=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   185
        return self.syssource.sqlexec(self.session, sql, args)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   186
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   187
    def execute(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   188
        """execute a plan and return resulting rows"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   189
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   190
            for step in self.steps:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   191
                result = step.execute()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   192
            # the latest executed step contains the full query result
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   193
            return result
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   194
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   195
            self.clean()
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   196
5168
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   197
    def make_temp_table_name(self, table):
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   198
        """
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   199
        return a temp table name according to db backend
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   200
        """
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   201
        return self.syssource.make_temp_table_name(table)
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   202
1ab032df5ca3 SQL Server port: temporary table handling
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5072
diff changeset
   203
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   204
    def init_temp_table(self, table, selected, sol):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   205
        """initialize sql schema and variable map for a temporary table which
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   206
        will be used to store result for the given rqlst
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   207
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   208
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   209
            outputmap, sqlschema, _ = self.temp_tables[table]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   210
            update_varmap(outputmap, selected, table)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   211
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   212
            sqlschema, outputmap = self.syssource.temp_table_def(selected, sol,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   213
                                                                 table)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   214
            self.temp_tables[table] = [outputmap, sqlschema, False]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   215
        return outputmap
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   216
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   217
    def create_temp_table(self, table):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   218
        """create a temporary table to store result for the given rqlst"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   219
        if not self.temp_tables[table][-1]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   220
            sqlschema = self.temp_tables[table][1]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   221
            self.syssource.create_temp_table(self.session, table, sqlschema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   222
            self.temp_tables[table][-1] = True
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   223
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   224
    def preprocess(self, union, security=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   225
        """insert security when necessary then annotate rql st for sql generation
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   226
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   227
        return rqlst to actually execute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   228
        """
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   229
        cached = None
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   230
        if security and self.session.read_security:
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   231
            # ensure security is turned of when security is inserted,
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   232
            # else we may loop for ever...
4954
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   233
            if self.session.transaction_data.get('security-rqlst-cache'):
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   234
                key = self.cache_key
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   235
            else:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   236
                key = None
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   237
            if key is not None and key in self.session.transaction_data:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   238
                cachedunion, args = self.session.transaction_data[key]
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   239
                union.children[:] = []
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   240
                for select in cachedunion.children:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   241
                    union.append(select)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   242
                union.has_text_query = cachedunion.has_text_query
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   243
                args.update(self.args)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   244
                self.args = args
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   245
                cached = True
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   246
            else:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   247
                noinvariant = set()
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   248
                with security_enabled(self.session, read=False):
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   249
                    self._insert_security(union, noinvariant)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   250
                if key is not None:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   251
                    self.session.transaction_data[key] = (union, self.args)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   252
        else:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   253
            noinvariant = ()
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   254
        if cached is None:
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   255
            self.rqlhelper.simplify(union)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   256
            self.sqlannotate(union)
96f67c5be0e6 [security] experimental rqlst cache for read security: to activate using a 'security-rqlst-cache' flag in transaction data when you'll issue a lot of identic queries with only kwargs varying
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4953
diff changeset
   257
            set_qdata(self.schema.rschema, union, noinvariant)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   258
        if union.has_text_query:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   259
            self.cache_key = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   260
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   261
    def _insert_security(self, union, noinvariant):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   262
        for select in union.children[:]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   263
            for subquery in select.with_:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   264
                self._insert_security(subquery.query, noinvariant)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   265
            localchecks, restricted = self._check_permissions(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   266
            if any(localchecks):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   267
                rewrite = self.session.rql_rewriter.rewrite
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   268
                nbtrees = len(localchecks)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   269
                myunion = union
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   270
                # transform in subquery when len(localchecks)>1 and groups
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   271
                if nbtrees > 1 and (select.orderby or select.groupby or
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   272
                                    select.having or select.has_aggregat or
5596
d66a5d98db5b [security] fix security insertion on DISTINCT queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   273
                                    select.distinct or
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   274
                                    select.limit or select.offset):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   275
                    newselect = Select()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   276
                    # only select variables in subqueries
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   277
                    origselection = select.selection
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   278
                    select.select_only_variables()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   279
                    select.has_aggregat = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   280
                    # create subquery first so correct node are used on copy
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   281
                    # (eg ColumnAlias instead of Variable)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   282
                    aliases = [VariableRef(newselect.get_variable(vref.name, i))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   283
                               for i, vref in enumerate(select.selection)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   284
                    selected = set(vref.name for vref in aliases)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   285
                    # now copy original selection and groups
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   286
                    for term in origselection:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   287
                        newselect.append_selected(term.copy(newselect))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   288
                    if select.orderby:
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   289
                        sortterms = []
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   290
                        for sortterm in select.orderby:
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   291
                            sortterms.append(sortterm.copy(newselect))
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   292
                            for fnode in sortterm.get_nodes(Function):
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   293
                                if fnode.name == 'FTIRANK':
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   294
                                    # we've to fetch the has_text relation as well
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   295
                                    var = fnode.children[0].variable
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   296
                                    rel = iter(var.stinfo['ftirels']).next()
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   297
                                    assert not rel.ored(), 'unsupported'
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   298
                                    newselect.add_restriction(rel.copy(newselect))
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   299
                                    # remove relation from the orig select and
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   300
                                    # cleanup variable stinfo
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   301
                                    rel.parent.remove(rel)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   302
                                    var.stinfo['ftirels'].remove(rel)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   303
                                    var.stinfo['relations'].remove(rel)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   304
                                    # XXX not properly re-annotated after security insertion?
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   305
                                    newvar = newselect.get_variable(var.name)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   306
                                    newvar.stinfo.setdefault('ftirels', set()).add(rel)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   307
                                    newvar.stinfo.setdefault('relations', set()).add(rel)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   308
                        newselect.set_orderby(sortterms)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   309
                        _expand_selection(select.orderby, selected, aliases, select, newselect)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   310
                        select.orderby = () # XXX dereference?
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   311
                    if select.groupby:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   312
                        newselect.set_groupby([g.copy(newselect) for g in select.groupby])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   313
                        _expand_selection(select.groupby, selected, aliases, select, newselect)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
                        select.groupby = () # XXX dereference?
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   315
                    if select.having:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   316
                        newselect.set_having([g.copy(newselect) for g in select.having])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   317
                        _expand_selection(select.having, selected, aliases, select, newselect)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   318
                        select.having = () # XXX dereference?
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   319
                    if select.limit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   320
                        newselect.limit = select.limit
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
                        select.limit = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   322
                    if select.offset:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
                        newselect.offset = select.offset
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
                        select.offset = 0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
                    myunion = Union()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
                    newselect.set_with([SubQuery(aliases, myunion)], check=False)
5596
d66a5d98db5b [security] fix security insertion on DISTINCT queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   327
                    newselect.distinct = select.distinct
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
                    solutions = [sol.copy() for sol in select.solutions]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
                    cleanup_solutions(newselect, solutions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   330
                    newselect.set_possible_types(solutions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
                    # if some solutions doesn't need rewriting, insert original
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   332
                    # select as first union subquery
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   333
                    if () in localchecks:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   334
                        myunion.append(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   335
                    # we're done, replace original select by the new select with
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   336
                    # subqueries (more added in the loop below)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   337
                    union.replace(select, newselect)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   338
                elif not () in localchecks:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   339
                    union.remove(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   340
                for lcheckdef, lchecksolutions in localchecks.iteritems():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   341
                    if not lcheckdef:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   342
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   343
                    myrqlst = select.copy(solutions=lchecksolutions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   344
                    myunion.append(myrqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
                    # in-place rewrite + annotation / simplification
4955
8ddd5e938804 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4954
diff changeset
   346
                    lcheckdef = [((var, 'X'), rqlexprs) for var, rqlexprs in lcheckdef]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   347
                    rewrite(myrqlst, lcheckdef, lchecksolutions, self.args)
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   348
                    add_noinvariant(noinvariant, restricted, myrqlst, nbtrees)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
                if () in localchecks:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   350
                    select.set_possible_types(localchecks[()])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   351
                    add_types_restriction(self.schema, select)
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   352
                    add_noinvariant(noinvariant, restricted, select, nbtrees)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   353
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   354
    def _check_permissions(self, rqlst):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   355
        """return a dict defining "local checks", e.g. RQLExpression defined in
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   356
        the schema that should be inserted in the original query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   357
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   358
        solutions where a variable has a type which the user can't definitly read
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   359
        are removed, else if the user may read it (eg if an rql expression is
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   360
        defined for the "read" permission of the related type), the local checks
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   361
        dict for the solution is updated
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   362
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   363
        return a dict with entries for each different local check necessary,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   364
        with associated solutions as value. A local check is defined by a list
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   365
        of 2-uple, with variable name as first item and the necessary rql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   366
        expression as second item for each variable which has to be checked.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   367
        So solutions which don't require local checks will be associated to
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   368
        the empty tuple key.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   369
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   370
        note: rqlst should not have been simplified at this point
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   371
        """
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   372
        session = self.session
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   373
        msgs = []
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   374
        neweids = session.transaction_data.get('neweids', ())
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   375
        varkwargs = {}
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   376
        if not session.transaction_data.get('security-rqlst-cache'):
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   377
            for var in rqlst.defined_vars.itervalues():
5004
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   378
                if var.stinfo['constnode'] is not None:
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   379
                    eid = var.stinfo['constnode'].eval(self.args)
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   380
                    varkwargs[var.name] = typed_eid(eid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   381
        # dictionnary of variables restricted for security reason
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   382
        localchecks = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   383
        restricted_vars = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   384
        newsolutions = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
        for solution in rqlst.solutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   386
            try:
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   387
                localcheck = check_read_access(session, rqlst, solution, self.args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   388
            except Unauthorized, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   389
                msg = 'remove %s from solutions since %s has no %s access to %s'
5419
0b7805928a27 [repo security] deal with rewriten constant nodes in check_read_access, necessary when repo is used as an external source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5168
diff changeset
   390
                msg %= (solution, session.user.login, ex.args[0], ex.args[1])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   391
                msgs.append(msg)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   392
                LOGGER.info(msg)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   393
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   394
                newsolutions.append(solution)
4953
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   395
                # try to benefit of rqlexpr.check cache for entities which
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   396
                # are specified by eid in query'args
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   397
                for varname, eid in varkwargs.iteritems():
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   398
                    try:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   399
                        rqlexprs = localcheck.pop(varname)
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   400
                    except KeyError:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   401
                        continue
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   402
                    if eid in neweids:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   403
                        continue
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   404
                    for rqlexpr in rqlexprs:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   405
                        if rqlexpr.check(session, eid):
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   406
                            break
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   407
                    else:
c8c0e10dbd97 [read security] minor optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4843
diff changeset
   408
                        raise Unauthorized()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   409
                restricted_vars.update(localcheck)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   410
                localchecks.setdefault(tuple(localcheck.iteritems()), []).append(solution)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   411
        # raise Unautorized exception if the user can't access to any solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   412
        if not newsolutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   413
            raise Unauthorized('\n'.join(msgs))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   414
        rqlst.set_possible_types(newsolutions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   415
        return localchecks, restricted_vars
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   416
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
    def finalize(self, select, solutions, insertedvars):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   418
        rqlst = Union()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   419
        rqlst.append(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   420
        for mainvarname, rschema, newvarname in insertedvars:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
            nvartype = str(rschema.objects(solutions[0][mainvarname])[0])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   422
            for sol in solutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
                sol[newvarname] = nvartype
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   424
        select.clean_solutions(solutions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   425
        self.rqlhelper.annotate(rqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   426
        self.preprocess(rqlst, security=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   427
        return rqlst
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   428
4759
af2e6c377c71 cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   429
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   430
class InsertPlan(ExecutionPlan):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   431
    """an execution model specific to the INSERT rql query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   432
    """
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   433
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   434
    def __init__(self, querier, rqlst, args, session):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   435
        ExecutionPlan.__init__(self, querier, rqlst, args, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   436
        # save originaly selected variable, we may modify this
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   437
        # dictionary for substitution (query parameters)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   438
        self.selected = rqlst.selection
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   439
        # list of new or updated entities definition (utils.Entity)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   440
        self.e_defs = [[]]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   441
        # list of new relation definition (3-uple (from_eid, r_type, to_eid)
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   442
        self.r_defs = set()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   443
        # indexes to track entity definitions bound to relation definitions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   444
        self._r_subj_index = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   445
        self._r_obj_index = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   446
        self._expanded_r_defs = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   447
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   448
    def add_entity_def(self, edef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   449
        """add an entity definition to build"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   450
        edef.querier_pending_relations = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   451
        self.e_defs[-1].append(edef)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   452
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   453
    def add_relation_def(self, rdef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   454
        """add an relation definition to build"""
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   455
        self.r_defs.add(rdef)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   456
        if not isinstance(rdef[0], int):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   457
            self._r_subj_index.setdefault(rdef[0], []).append(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   458
        if not isinstance(rdef[2], int):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   459
            self._r_obj_index.setdefault(rdef[2], []).append(rdef)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   460
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   461
    def substitute_entity_def(self, edef, edefs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   462
        """substitute an incomplete entity definition by a list of complete
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   463
        equivalents
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   464
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   465
        e.g. on queries such as ::
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   466
          INSERT Personne X, Societe Y: X nom N, Y nom 'toto', X travaille Y
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   467
          WHERE U login 'admin', U login N
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   468
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   469
        X will be inserted as many times as U exists, and so the X travaille Y
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   470
        relations as to be added as many time as X is inserted
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   471
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   472
        if not edefs or not self.e_defs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   473
            # no result, no entity will be created
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   474
            self.e_defs = ()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   475
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   476
        # first remove the incomplete entity definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   477
        colidx = self.e_defs[0].index(edef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   478
        for i, row in enumerate(self.e_defs[:]):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   479
            self.e_defs[i][colidx] = edefs[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   480
            samplerow = self.e_defs[i]
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   481
            for edef_ in edefs[1:]:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   482
                row = samplerow[:]
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   483
                row[colidx] = edef_
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   484
                self.e_defs.append(row)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   485
        # now, see if this entity def is referenced as subject in some relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   486
        # definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   487
        if self._r_subj_index.has_key(edef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   488
            for rdef in self._r_subj_index[edef]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   489
                expanded = self._expanded(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   490
                result = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   491
                for exp_rdef in expanded:
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   492
                    for edef_ in edefs:
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   493
                        result.append( (edef_, exp_rdef[1], exp_rdef[2]) )
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   494
                self._expanded_r_defs[rdef] = result
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   495
        # and finally, see if this entity def is referenced as object in some
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   496
        # relation definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   497
        if self._r_obj_index.has_key(edef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   498
            for rdef in self._r_obj_index[edef]:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   499
                expanded = self._expanded(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   500
                result = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   501
                for exp_rdef in expanded:
5657
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   502
                    for edef_ in edefs:
8f56691f01de [querier] test and fix potential bug in insertion query with multiple relations inside, caused by a name conflict. Also fix another bug non reproduced by using a set instead of a list (list may leads to compare entity instances which isn't allowed).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5596
diff changeset
   503
                        result.append( (exp_rdef[0], exp_rdef[1], edef_) )
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   504
                self._expanded_r_defs[rdef] = result
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   505
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   506
    def _expanded(self, rdef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   507
        """return expanded value for the given relation definition"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   508
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   509
            return self._expanded_r_defs[rdef]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   510
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   511
            self.r_defs.remove(rdef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   512
            return [rdef]
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   513
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   514
    def relation_defs(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   515
        """return the list for relation definitions to insert"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   516
        for rdefs in self._expanded_r_defs.values():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   517
            for rdef in rdefs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   518
                yield rdef
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   519
        for rdef in self.r_defs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   520
            yield rdef
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   521
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   522
    def insert_entity_defs(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   523
        """return eids of inserted entities in a suitable form for the resulting
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   524
        result set, e.g.:
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   525
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   526
        e.g. on queries such as ::
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   527
          INSERT Personne X, Societe Y: X nom N, Y nom 'toto', X travaille Y
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   528
          WHERE U login 'admin', U login N
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   529
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   530
        if there is two entities matching U, the result set will look like
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   531
        [(eidX1, eidY1), (eidX2, eidY2)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   532
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   533
        session = self.session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   534
        repo = session.repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   535
        results = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   536
        for row in self.e_defs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   537
            results.append([repo.glob_add_entity(session, edef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   538
                            for edef in row])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   539
        return results
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   540
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   541
    def insert_relation_defs(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   542
        session = self.session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   543
        repo = session.repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   544
        for subj, rtype, obj in self.relation_defs():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   545
            # if a string is given into args instead of an int, we get it here
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   546
            if isinstance(subj, basestring):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   547
                subj = typed_eid(subj)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   548
            elif not isinstance(subj, (int, long)):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   549
                subj = subj.eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   550
            if isinstance(obj, basestring):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   551
                obj = typed_eid(obj)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   552
            elif not isinstance(obj, (int, long)):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   553
                obj = obj.eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   554
            if repo.schema.rschema(rtype).inlined:
2680
66472d85d548 [R] use req.entity_from_eid
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2646
diff changeset
   555
                entity = session.entity_from_eid(subj)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   556
                entity[rtype] = obj
3171
b10f0c1f3fe4 [repo] oops, glob_update_entity takes a list of edited attributes has third argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3109
diff changeset
   557
                repo.glob_update_entity(session, entity, set((rtype,)))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   558
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   559
                repo.glob_add_relation(session, subj, rtype, obj)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   560
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   561
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   562
class QuerierHelper(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   563
    """helper class to execute rql queries, putting all things together"""
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   564
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   565
    def __init__(self, repo, schema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   566
        # system info helper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   567
        self._repo = repo
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2101
diff changeset
   568
        # instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   569
        self.set_schema(schema)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   570
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   571
    def set_schema(self, schema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   572
        self.schema = schema
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   573
        repo = self._repo
5176
ddd5219d7eef [querier] don't need a Cache instance for rql cacke key cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5174
diff changeset
   574
        # rql st and solution cache. Don't bother using a Cache instance: we
ddd5219d7eef [querier] don't need a Cache instance for rql cacke key cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5174
diff changeset
   575
        # should have a limited number of queries in there, since there are no
ddd5219d7eef [querier] don't need a Cache instance for rql cacke key cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5174
diff changeset
   576
        # entries in this cache for user queries (which have no args)
ddd5219d7eef [querier] don't need a Cache instance for rql cacke key cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5174
diff changeset
   577
        self._rql_cache = {}
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   578
        # rql cache key cache
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   579
        self._rql_ck_cache = Cache(repo.config['rql-cache-size'])
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   580
        # some cache usage stats
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   581
        self.cache_hit, self.cache_miss = 0, 0
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   582
        # rql parsing / analysing helper
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   583
        self.solutions = repo.vreg.solutions
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   584
        rqlhelper = repo.vreg.rqlhelper
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   585
        # set backend on the rql helper, will be used for function checking
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5726
diff changeset
   586
        rqlhelper.backend = repo.config.sources()['system']['db-driver']
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   587
        self._parse = rqlhelper.parse
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   588
        self._annotate = rqlhelper.annotate
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   589
        # rql planner
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   590
        # note: don't use repo.sources, may not be built yet, and also "admin"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   591
        #       isn't an actual source
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   592
        if len([uri for uri in repo.config.sources() if uri != 'admin']) < 2:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   593
            from cubicweb.server.ssplanner import SSPlanner
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   594
            self._planner = SSPlanner(schema, rqlhelper)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   595
        else:
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   596
            from cubicweb.server.msplanner import MSPlanner
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   597
            self._planner = MSPlanner(schema, rqlhelper)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   598
        # sql generation annotator
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   599
        self.sqlgen_annotate = SQLGenAnnotator(schema).annotate
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   600
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   601
    def parse(self, rql, annotate=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   602
        """return a rql syntax tree for the given rql"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   603
        try:
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   604
            return self._parse(unicode(rql), annotate=annotate)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   605
        except UnicodeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   606
            raise RQLSyntaxError(rql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   607
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   608
    def plan_factory(self, rqlst, args, session):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   609
        """create an execution plan for an INSERT RQL query"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   610
        if rqlst.TYPE == 'insert':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   611
            return InsertPlan(self, rqlst, args, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   612
        return ExecutionPlan(self, rqlst, args, session)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   613
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   614
    def execute(self, session, rql, args=None, build_descr=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   615
        """execute a rql query, return resulting rows and their description in
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   616
        a `ResultSet` object
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   617
4654
717310b3d576 docstring improvement
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 4533
diff changeset
   618
        * `rql` should be an Unicode string or a plain ASCII string
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   619
        * `args` the optional parameters dictionary associated to the query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   620
        * `build_descr` is a boolean flag indicating if the description should
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   621
          be built on select queries (if false, the description will be en empty
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   622
          list)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   623
4654
717310b3d576 docstring improvement
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 4533
diff changeset
   624
        on INSERT queries, there will be one row with the eid of each inserted
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   625
        entity
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1133
diff changeset
   626
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   627
        result for DELETE and SET queries is undefined yet
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   628
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   629
        to maximize the rql parsing/analyzing cache performance, you should
4654
717310b3d576 docstring improvement
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 4533
diff changeset
   630
        always use substitute arguments in queries (i.e. avoid query such as
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   631
        'Any X WHERE X eid 123'!)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   632
        """
2621
1b9d08840a0e R [querier] debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2601
diff changeset
   633
        if server.DEBUG & (server.DBG_RQL | server.DBG_SQL):
2629
0d445c2171e4 R [querier] debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2621
diff changeset
   634
            if server.DEBUG & (server.DBG_MORE | server.DBG_SQL):
2621
1b9d08840a0e R [querier] debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2601
diff changeset
   635
                print '*'*80
2646
d2874ddd4347 #F [querier] debugging tweak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2629
diff changeset
   636
            print 'querier input', rql, args
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   637
        # parse the query and binds variables
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   638
        try:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   639
            cachekey = rql
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   640
            if args:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   641
                eidkeys = self._rql_ck_cache[rql]
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   642
                if eidkeys:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   643
                    try:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   644
                        cachekey = self._repo.querier_cache_key(session, rql,
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   645
                                                                args, eidkeys)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   646
                    except UnknownEid:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   647
                        # we want queries such as "Any X WHERE X eid 9999"
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   648
                        # return an empty result instead of raising UnknownEid
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   649
                        return empty_rset(rql, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   650
            rqlst = self._rql_cache[cachekey]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   651
            self.cache_hit += 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   652
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   653
            self.cache_miss += 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   654
            rqlst = self.parse(rql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   655
            try:
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   656
                eidkeys = self.solutions(session, rqlst, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   657
            except UnknownEid:
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   658
                # we want queries such as "Any X WHERE X eid 9999" return an
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   659
                # empty result instead of raising UnknownEid
4722
9c13d5db03d9 pylint suggested refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   660
                return empty_rset(rql, args, rqlst)
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   661
            self._rql_ck_cache[rql] = eidkeys
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   662
            if eidkeys:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   663
                cachekey = self._repo.querier_cache_key(session, rql, args,
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   664
                                                        eidkeys)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   665
            self._rql_cache[cachekey] = rqlst
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   666
        orig_rqlst = rqlst
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4957
diff changeset
   667
        if rqlst.TYPE != 'select':
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   668
            if session.read_security:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   669
                check_no_password_selected(rqlst)
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   670
            # write query, ensure session's mode is 'write' so connections won't
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
   671
            # be released until commit/rollback
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   672
            session.mode = 'write'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   673
            cachekey = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   674
        else:
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   675
            if session.read_security:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   676
                for select in rqlst.children:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   677
                    check_no_password_selected(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   678
            # on select query, always copy the cached rqlst so we don't have to
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   679
            # bother modifying it. This is not necessary on write queries since
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   680
            # a new syntax tree is built from them.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   681
            rqlst = rqlst.copy()
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3171
diff changeset
   682
            self._annotate(rqlst)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   683
        # make an execution plan
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   684
        plan = self.plan_factory(rqlst, args, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   685
        plan.cache_key = cachekey
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   686
        self._planner.build_plan(plan)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   687
        # execute the plan
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   688
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   689
            results = plan.execute()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   690
        except Unauthorized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   691
            # XXX this could be done in security's after_add_relation hooks
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   692
            # since it's actually realy only needed there (other relations
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   693
            # security is done *before* actual changes, and add/update entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   694
            # security is done after changes but in an operation, and exception
4533
05eadf88b797 rephrase
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4532
diff changeset
   695
            # generated in operation's events properly generate a rollback on
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   696
            # the session). Even though, this is done here for a better
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   697
            # consistency: getting an Unauthorized exception means the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   698
            # transaction has been rollbacked
4532
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   699
            #
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   700
            # notes:
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   701
            # * we should not reset the pool here, since we don't want the
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   702
            #   session to loose its pool during processing
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   703
            # * don't rollback if we're in the commit process, will be handled
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   704
            #   by the session
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   705
            if session.commit_state is None:
85116e75f561 fix auto-rollback on unauthorized error:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   706
                session.rollback(reset_pool=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   707
            raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   708
        # build a description for the results if necessary
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   709
        descr = ()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   710
        if build_descr:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   711
            if rqlst.TYPE == 'select':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   712
                # sample selection
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   713
                descr = session.build_description(orig_rqlst, args, results)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   714
            elif rqlst.TYPE == 'insert':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   715
                # on insert plan, some entities may have been auto-casted,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   716
                # so compute description manually even if there is only
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   717
                # one solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   718
                basedescr = [None] * len(plan.selected)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   719
                todetermine = zip(xrange(len(plan.selected)), repeat(False))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   720
                descr = session._build_descr(results, basedescr, todetermine)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   721
            # FIXME: get number of affected entities / relations on non
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   722
            # selection queries ?
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   723
        # return a result set object
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5082
diff changeset
   724
        return ResultSet(results, rql, args, descr, orig_rqlst)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   725
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   726
from logging import getLogger
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   727
from cubicweb import set_log_methods
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   728
LOGGER = getLogger('cubicweb.querier')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   729
set_log_methods(QuerierHelper, LOGGER)