server/ssplanner.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Mon, 08 Mar 2010 17:57:29 +0100
changeset 4831 c5aec27c1bf7
parent 4795 f1c8bc628b45
child 4835 13b0b96d7982
permissions -rw-r--r--
[repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     1
"""plan execution of rql queries on a single source
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     2
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     3
:organization: Logilab
4212
ab6573088b4a update copyright: welcome 2010
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3689
diff changeset
     4
:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     5
:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr
1977
606923dff11b big bunch of copyright / docstring update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
     6
:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     7
"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     8
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     9
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    10
from copy import copy
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    11
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    12
from rql.stmts import Union, Select
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    13
from rql.nodes import Constant, Relation
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    14
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    15
from cubicweb import QueryError, typed_eid
2596
d02eed70937f [R repo, schema] use VIRTUAL_RTYPES const
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2101
diff changeset
    16
from cubicweb.schema import VIRTUAL_RTYPES
3437
a30b5b5138a4 cw.rqlrewrite shouldn't depend on cw.server
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2921
diff changeset
    17
from cubicweb.rqlrewrite import add_types_restriction
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    19
READ_ONLY_RTYPES = set(('eid', 'has_text', 'is', 'is_instance_of', 'identity'))
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    20
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    21
_CONSTANT = object()
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    22
_FROM_SUBSTEP = object()
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    23
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    24
def _extract_const_attributes(plan, rqlst, to_build):
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    25
    """add constant values to entity def, mark variables to be selected
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    26
    """
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    27
    to_select = {}
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    28
    for relation in rqlst.main_relations:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    29
        lhs, rhs = relation.get_variable_parts()
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    30
        rtype = relation.r_type
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    31
        if rtype in READ_ONLY_RTYPES:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    32
            raise QueryError("can't assign to %s" % rtype)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    33
        try:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    34
            edef = to_build[str(lhs)]
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    35
        except KeyError:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    36
            # lhs var is not to build, should be selected and added as an
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    37
            # object relation
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    38
            edef = to_build[str(rhs)]
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    39
            to_select.setdefault(edef, []).append((rtype, lhs, 1))
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    40
        else:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    41
            if isinstance(rhs, Constant) and not rhs.uid:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    42
                # add constant values to entity def
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    43
                value = rhs.eval(plan.args)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    44
                eschema = edef.e_schema
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    45
                attrtype = eschema.subjrels[rtype].objects(eschema)[0]
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    46
                if attrtype == 'Password' and isinstance(value, unicode):
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    47
                    value = value.encode('UTF8')
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    48
                edef[rtype] = value
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    49
            elif to_build.has_key(str(rhs)):
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    50
                # create a relation between two newly created variables
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    51
                plan.add_relation_def((edef, rtype, to_build[rhs.name]))
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    52
            else:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    53
                to_select.setdefault(edef, []).append( (rtype, rhs, 0) )
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    54
    return to_select
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    55
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    56
def _extract_eid_consts(plan, rqlst):
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    57
    """return a dict mapping rqlst variable object to their eid if specified in
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    58
    the syntax tree
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    59
    """
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    60
    session = plan.session
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    61
    eschema = session.vreg.schema.eschema
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    62
    if rqlst.where is None:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    63
        return {}
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    64
    eidconsts = {}
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    65
    neweids = session.transaction_data.get('neweids', ())
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    66
    for rel in rqlst.where.get_nodes(Relation):
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    67
        if rel.r_type == 'eid' and not rel.neged(strict=True):
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    68
            lhs, rhs = rel.get_variable_parts()
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    69
            if isinstance(rhs, Constant):
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    70
                eid = typed_eid(rhs.eval(plan.args))
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    71
                # check read permission here since it may not be done by
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    72
                # the generated select substep if not emited (eg nothing
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    73
                # to be selected)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    74
                if eid not in neweids:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    75
                    eschema(session.describe(eid)[0]).check_perm(session, 'read')
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    76
                eidconsts[lhs.variable] = eid
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    77
    return eidconsts
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    78
4795
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    79
def _build_substep_query(select, origrqlst):
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    80
    """Finalize substep select query that should be executed to get proper
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    81
    selection of stuff to insert/update.
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    82
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    83
    Return None when no query actually needed, else the given select node that
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    84
    will be used as substep query.
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    85
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    86
    When select has nothing selected, search in origrqlst for restriction that
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    87
    should be considered.
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    88
    """
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    89
    if select.selection:
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    90
        if origrqlst.where is not None:
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    91
            select.set_where(origrqlst.where.copy(select))
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    92
        return select
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    93
    if origrqlst.where is None:
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    94
        return
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    95
    for rel in origrqlst.where.iget_nodes(Relation):
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    96
        # search for a relation which is neither a type restriction (is) nor an
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    97
        # eid specification (not neged eid with constant node
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    98
        if rel.neged(strict=True) or not (
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
    99
            rel.is_types_restriction() or
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   100
            (rel.r_type == 'eid'
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   101
             and isinstance(rel.get_variable_parts()[1], Constant))):
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   102
            break
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   103
    else:
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   104
        return
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   105
    select.set_where(origrqlst.where.copy(select))
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   106
    if not select.selection:
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   107
        # no selection, append one randomly
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   108
        select.append_selected(rel.children[0].copy(select))
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   109
    return select
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   110
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   111
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   112
class SSPlanner(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   113
    """SingleSourcePlanner: build execution plan for rql queries
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   114
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   115
    optimized for single source repositories
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   116
    """
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   117
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   118
    def __init__(self, schema, rqlhelper):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   119
        self.schema = schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   120
        self.rqlhelper = rqlhelper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   121
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   122
    def build_plan(self, plan):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   123
        """build an execution plan from a RQL query
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   124
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   125
        do nothing here, dispatch according to the statement type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   126
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   127
        build_plan = getattr(self, 'build_%s_plan' % plan.rqlst.TYPE)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   128
        for step in build_plan(plan, plan.rqlst):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
            plan.add_step(step)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   130
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   131
    def build_select_plan(self, plan, rqlst):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   132
        """build execution plan for a SELECT RQL query. Suppose only one source
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   133
        is available and so avoid work need for query decomposition among sources
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   134
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
        the rqlst should not be tagged at this point.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
        plan.preprocess(rqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   138
        return (OneFetchStep(plan, rqlst, plan.session.repo.sources),)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   139
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   140
    def build_insert_plan(self, plan, rqlst):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
        """get an execution plan from an INSERT RQL query"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
        # each variable in main variables is a new entity to insert
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   143
        to_build = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   144
        session = plan.session
2650
18aec79ec3a3 R [vreg] important refactoring of the vregistry, moving behaviour to end dictionnary (and so leaving room for more flexibility ; keep bw compat ; update api usage in cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
   145
        etype_class = session.vreg['etypes'].etype_class
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   146
        for etype, var in rqlst.main_variables:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   147
            # need to do this since entity class is shared w. web client code !
2650
18aec79ec3a3 R [vreg] important refactoring of the vregistry, moving behaviour to end dictionnary (and so leaving room for more flexibility ; keep bw compat ; update api usage in cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
   148
            to_build[var.name] = etype_class(etype)(session)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   149
            plan.add_entity_def(to_build[var.name])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   150
        # add constant values to entity def, mark variables to be selected
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   151
        to_select = _extract_const_attributes(plan, rqlst, to_build)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   152
        # add necessary steps to add relations and update attributes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   153
        step = InsertStep(plan) # insert each entity and its relations
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   154
        step.children += self._compute_relation_steps(plan, rqlst, to_select)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   155
        return (step,)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   156
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   157
    def _compute_relation_steps(self, plan, rqlst, to_select):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   158
        """handle the selection of relations for an insert query"""
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   159
        eidconsts = _extract_eid_consts(plan, rqlst)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
        for edef, rdefs in to_select.items():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   161
            # create a select rql st to fetch needed data
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   162
            select = Select()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   163
            eschema = edef.e_schema
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   164
            for i, (rtype, term, reverse) in enumerate(rdefs):
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   165
                if getattr(term, 'variable', None) in eidconsts:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   166
                    value = eidconsts[term.variable]
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   167
                else:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   168
                    select.append_selected(term.copy(select))
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   169
                    value = _FROM_SUBSTEP
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   170
                if reverse:
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   171
                    rdefs[i] = (rtype, InsertRelationsStep.REVERSE_RELATION, value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   172
                else:
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3437
diff changeset
   173
                    rschema = eschema.subjrels[rtype]
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3437
diff changeset
   174
                    if rschema.final or rschema.inlined:
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   175
                        rdefs[i] = (rtype, InsertRelationsStep.FINAL, value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   176
                    else:
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   177
                        rdefs[i] = (rtype, InsertRelationsStep.RELATION, value)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   178
            step = InsertRelationsStep(plan, edef, rdefs)
4795
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   179
            select = _build_substep_query(select, rqlst)
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   180
            if select is not None:
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   181
                step.children += self._select_plan(plan, select, rqlst.solutions)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   182
            yield step
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   183
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   184
    def build_delete_plan(self, plan, rqlst):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   185
        """get an execution plan from a DELETE RQL query"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   186
        # build a select query to fetch entities to delete
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   187
        steps = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   188
        for etype, var in rqlst.main_variables:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   189
            step = DeleteEntitiesStep(plan)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   190
            step.children += self._sel_variable_step(plan, rqlst.solutions,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   191
                                                     rqlst.where, etype, var)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   192
            steps.append(step)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   193
        for relation in rqlst.main_relations:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   194
            step = DeleteRelationsStep(plan, relation.r_type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   195
            step.children += self._sel_relation_steps(plan, rqlst.solutions,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   196
                                                      rqlst.where, relation)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   197
            steps.append(step)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   198
        return steps
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   199
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   200
    def _sel_variable_step(self, plan, solutions, restriction, etype, varref):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   201
        """handle the selection of variables for a delete query"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   202
        select = Select()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   203
        varref = varref.copy(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   204
        select.defined_vars = {varref.name: varref.variable}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   205
        select.append_selected(varref)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   206
        if restriction is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   207
            select.set_where(restriction.copy(select))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   208
        if etype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   209
            select.add_type_restriction(varref.variable, etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   210
        return self._select_plan(plan, select, solutions)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   211
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   212
    def _sel_relation_steps(self, plan, solutions, restriction, relation):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   213
        """handle the selection of relations for a delete query"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   214
        select = Select()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   215
        lhs, rhs = relation.get_variable_parts()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   216
        select.append_selected(lhs.copy(select))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   217
        select.append_selected(rhs.copy(select))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   218
        select.set_where(relation.copy(select))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   219
        if restriction is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   220
            select.add_restriction(restriction.copy(select))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   221
        return self._select_plan(plan, select, solutions)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   222
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   223
    def build_set_plan(self, plan, rqlst):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   224
        """get an execution plan from an SET RQL query"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   225
        getrschema = self.schema.rschema
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   226
        select = Select()   # potential substep query
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   227
        selectedidx = {}    # local state
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   228
        attributes = set()  # edited attributes
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   229
        updatedefs = []     # definition of update attributes/relations
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   230
        selidx = residx = 0 # substep selection / resulting rset indexes
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   231
        # search for eid const in the WHERE clause
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   232
        eidconsts = _extract_eid_consts(plan, rqlst)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   233
        # build `updatedefs` describing things to update and add necessary
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   234
        # variables to the substep selection
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   235
        for i, relation in enumerate(rqlst.main_relations):
2596
d02eed70937f [R repo, schema] use VIRTUAL_RTYPES const
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2101
diff changeset
   236
            if relation.r_type in VIRTUAL_RTYPES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   237
                raise QueryError('can not assign to %r relation'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   238
                                 % relation.r_type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   239
            lhs, rhs = relation.get_variable_parts()
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   240
            lhskey = lhs.as_string('utf-8')
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   241
            if not lhskey in selectedidx:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   242
                if lhs.variable in eidconsts:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   243
                    eid = eidconsts[lhs.variable]
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   244
                    lhsinfo = (_CONSTANT, eid, residx)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   245
                else:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   246
                    select.append_selected(lhs.copy(select))
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   247
                    lhsinfo = (_FROM_SUBSTEP, selidx, residx)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   248
                    selidx += 1
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   249
                residx += 1
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   250
                selectedidx[lhskey] = lhsinfo
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   251
            else:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   252
                lhsinfo = selectedidx[lhskey][:-1] + (None,)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   253
            rhskey = rhs.as_string('utf-8')
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   254
            if not rhskey in selectedidx:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   255
                if isinstance(rhs, Constant):
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   256
                    rhsinfo = (_CONSTANT, rhs.eval(plan.args), residx)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   257
                elif getattr(rhs, 'variable', None) in eidconsts:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   258
                    eid = eidconsts[rhs.variable]
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   259
                    rhsinfo = (_CONSTANT, eid, residx)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   260
                else:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   261
                    select.append_selected(rhs.copy(select))
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   262
                    rhsinfo = (_FROM_SUBSTEP, selidx, residx)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   263
                    selidx += 1
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   264
                residx += 1
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   265
                selectedidx[rhskey] = rhsinfo
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   266
            else:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   267
                rhsinfo = selectedidx[rhskey][:-1] + (None,)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   268
            rschema = getrschema(relation.r_type)
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   269
            updatedefs.append( (lhsinfo, rhsinfo, rschema) )
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3437
diff changeset
   270
            if rschema.final or rschema.inlined:
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   271
                attributes.add(relation.r_type)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   272
        # the update step
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   273
        step = UpdateStep(plan, updatedefs, attributes)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   274
        # when necessary add substep to fetch yet unknown values
4795
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   275
        select = _build_substep_query(select, rqlst)
f1c8bc628b45 [repo] fix bug introduced by 4757:ec9c20c6b9f7, testing for select.selection is not enough to avoid the substep query, we should check there is no interesting restriction (test added)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4764
diff changeset
   276
        if select is not None:
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   277
            # set distinct to avoid potential duplicate key error
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   278
            select.distinct = True
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   279
            step.children += self._select_plan(plan, select, rqlst.solutions)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   280
        return (step,)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   281
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   282
    # internal methods ########################################################
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   283
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   284
    def _select_plan(self, plan, select, solutions):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   285
        union = Union()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   286
        union.append(select)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   287
        select.clean_solutions(solutions)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   288
        add_types_restriction(self.schema, select)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   289
        self.rqlhelper.annotate(union)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
        return self.build_select_plan(plan, union)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   291
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   292
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   293
# execution steps and helper functions ########################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   294
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   295
def varmap_test_repr(varmap, tablesinorder):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   296
    if varmap is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   297
        return varmap
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   298
    maprepr = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   299
    for var, sql in varmap.iteritems():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   300
        table, col = sql.split('.')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   301
        maprepr[var] = '%s.%s' % (tablesinorder[table], col)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   302
    return maprepr
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   303
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   304
def offset_result(offset, result):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   305
    offset -= len(result)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   306
    if offset < 0:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   307
        result = result[offset:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   308
        offset = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   309
    elif offset == 0:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   310
        offset = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   311
        result = ()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   312
    return offset, result
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   313
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   315
class LimitOffsetMixIn(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   316
    limit = offset = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   317
    def set_limit_offset(self, limit, offset):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   318
        self.limit = limit
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   319
        self.offset = offset or None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   320
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   321
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   322
class Step(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
    """base abstract class for execution step"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
    def __init__(self, plan):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
        self.plan = plan
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
        self.children = []
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   327
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
    def execute_child(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
        assert len(self.children) == 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   330
        return self.children[0].execute()
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   331
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   332
    def execute_children(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   333
        for step in self.children:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   334
            step.execute()
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   335
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   336
    def execute(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   337
        """execute this step and store partial (eg this step) results"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   338
        raise NotImplementedError()
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   339
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   340
    def mytest_repr(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   341
        """return a representation of this step suitable for test"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   342
        return (self.__class__.__name__,)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   343
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   344
    def test_repr(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
        """return a representation of this step suitable for test"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   346
        return self.mytest_repr() + (
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   347
            [step.test_repr() for step in self.children],)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   348
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   349
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   350
class OneFetchStep(LimitOffsetMixIn, Step):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   351
    """step consisting in fetching data from sources and directly returning
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   352
    results
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   353
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   354
    def __init__(self, plan, union, sources, inputmap=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   355
        Step.__init__(self, plan)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   356
        self.union = union
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   357
        self.sources = sources
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   358
        self.inputmap = inputmap
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   359
        self.set_limit_offset(union.children[-1].limit, union.children[-1].offset)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   360
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   361
    def set_limit_offset(self, limit, offset):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   362
        LimitOffsetMixIn.set_limit_offset(self, limit, offset)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   363
        for select in self.union.children:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   364
            select.limit = limit
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   365
            select.offset = offset
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   366
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   367
    def execute(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   368
        """call .syntax_tree_search with the given syntax tree on each
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   369
        source for each solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   370
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   371
        self.execute_children()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   372
        session = self.plan.session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   373
        args = self.plan.args
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   374
        inputmap = self.inputmap
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   375
        union = self.union
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   376
        # do we have to use a inputmap from a previous step ? If so disable
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   377
        # cachekey
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   378
        if inputmap or self.plan.cache_key is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   379
            cachekey = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   380
        # union may have been splited into subqueries, rebuild a cache key
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   381
        elif isinstance(self.plan.cache_key, tuple):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   382
            cachekey = list(self.plan.cache_key)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   383
            cachekey[0] = union.as_string()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   384
            cachekey = tuple(cachekey)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   386
            cachekey = union.as_string()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   387
        result = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   388
        # limit / offset processing
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   389
        limit = self.limit
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   390
        offset = self.offset
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   391
        if offset is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   392
            if len(self.sources) > 1:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   393
                # we'll have to deal with limit/offset by ourself
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   394
                if union.children[-1].limit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   395
                    union.children[-1].limit = limit + offset
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   396
                union.children[-1].offset = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   397
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   398
                offset, limit = None, None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   399
        for source in self.sources:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   400
            if offset is None and limit is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
                # modifying the sample rqlst is enough since sql generation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   402
                # will pick it here as well
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   403
                union.children[-1].limit = limit - len(result)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   404
            result_ = source.syntax_tree_search(session, union, args, cachekey,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   405
                                                inputmap)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   406
            if offset is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   407
                offset, result_ = offset_result(offset, result_)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   408
            result += result_
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   409
            if limit is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   410
                if len(result) >= limit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   411
                    return result[:limit]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   412
        #print 'ONEFETCH RESULT %s' % (result)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   413
        return result
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   414
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   415
    def mytest_repr(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   416
        """return a representation of this step suitable for test"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   418
            inputmap = varmap_test_repr(self.inputmap, self.plan.tablesinorder)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   419
        except AttributeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   420
            inputmap = self.inputmap
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
        return (self.__class__.__name__,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   422
                sorted((r.as_string(kwargs=self.plan.args), r.solutions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
                       for r in self.union.children),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   424
                self.limit, self.offset,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   425
                sorted(self.sources), inputmap)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   426
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   427
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   428
# UPDATE/INSERT/DELETE steps ##################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   429
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   430
class InsertRelationsStep(Step):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   431
    """step consisting in adding attributes/relations to entity defs from a
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   432
    previous FetchStep
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   433
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   434
    relations values comes from the latest result, with one columns for
2921
8e2544e78a5e test and fix rset returned by SET query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2875
diff changeset
   435
    each relation defined in self.rdefs
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   436
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   437
    for one entity definition, we'll construct N entity, where N is the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   438
    number of the latest result
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   439
    """
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   440
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   441
    FINAL = 0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   442
    RELATION = 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   443
    REVERSE_RELATION = 2
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   444
2921
8e2544e78a5e test and fix rset returned by SET query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2875
diff changeset
   445
    def __init__(self, plan, edef, rdefs):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   446
        Step.__init__(self, plan)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   447
        # partial entity definition to expand
2921
8e2544e78a5e test and fix rset returned by SET query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2875
diff changeset
   448
        self.edef = edef
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   449
        # definition of relations to complete
2921
8e2544e78a5e test and fix rset returned by SET query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2875
diff changeset
   450
        self.rdefs = rdefs
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   451
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   452
    def execute(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   453
        """execute this step"""
2921
8e2544e78a5e test and fix rset returned by SET query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2875
diff changeset
   454
        base_edef = self.edef
8e2544e78a5e test and fix rset returned by SET query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2875
diff changeset
   455
        edefs = []
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   456
        if self.children:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   457
            result = self.execute_child()
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   458
        else:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   459
            result = [[]]
2921
8e2544e78a5e test and fix rset returned by SET query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2875
diff changeset
   460
        for row in result:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   461
            # get a new entity definition for this row
2921
8e2544e78a5e test and fix rset returned by SET query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2875
diff changeset
   462
            edef = copy(base_edef)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   463
            # complete this entity def using row values
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   464
            index = 0
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   465
            for rtype, rorder, value in self.rdefs:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   466
                if value is _FROM_SUBSTEP:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   467
                    value = row[index]
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   468
                    index += 1
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   469
                if rorder == InsertRelationsStep.FINAL:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   470
                    edef[rtype] = value
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   471
                elif rorder == InsertRelationsStep.RELATION:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   472
                    self.plan.add_relation_def( (edef, rtype, value) )
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   473
                    edef.querier_pending_relations[(rtype, 'subject')] = value
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   474
                else:
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   475
                    self.plan.add_relation_def( (value, rtype, edef) )
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   476
                    edef.querier_pending_relations[(rtype, 'object')] = value
2921
8e2544e78a5e test and fix rset returned by SET query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2875
diff changeset
   477
            edefs.append(edef)
8e2544e78a5e test and fix rset returned by SET query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2875
diff changeset
   478
        self.plan.substitute_entity_def(base_edef, edefs)
8e2544e78a5e test and fix rset returned by SET query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2875
diff changeset
   479
        return result
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   480
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   481
class InsertStep(Step):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   482
    """step consisting in inserting new entities / relations"""
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   483
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   484
    def execute(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   485
        """execute this step"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   486
        for step in self.children:
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   487
            assert isinstance(step, InsertRelationsStep)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   488
            step.plan = self.plan
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   489
            step.execute()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   490
        # insert entities first
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   491
        result = self.plan.insert_entity_defs()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   492
        # then relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   493
        self.plan.insert_relation_defs()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   494
        # return eids of inserted entities
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   495
        return result
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   496
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   497
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   498
class DeleteEntitiesStep(Step):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   499
    """step consisting in deleting entities"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   500
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   501
    def execute(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   502
        """execute this step"""
3648
665c37544060 on entity deletion query, return eids of deleted entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3437
diff changeset
   503
        results = self.execute_child()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   504
        todelete = frozenset(typed_eid(eid) for eid, in self.execute_child())
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   505
        session = self.plan.session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   506
        delete = session.repo.glob_delete_entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   507
        # register pending eids first to avoid multiple deletion
2101
08003e0354a7 update transaction data api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   508
        pending = session.transaction_data.setdefault('pendingeids', set())
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   509
        actual = todelete - pending
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   510
        pending |= actual
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   511
        for eid in actual:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   512
            delete(session, eid)
3648
665c37544060 on entity deletion query, return eids of deleted entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3437
diff changeset
   513
        return results
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   514
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   515
class DeleteRelationsStep(Step):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   516
    """step consisting in deleting relations"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   517
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   518
    def __init__(self, plan, rtype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   519
        Step.__init__(self, plan)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   520
        self.rtype = rtype
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   521
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   522
    def execute(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   523
        """execute this step"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   524
        session = self.plan.session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   525
        delete = session.repo.glob_delete_relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   526
        for subj, obj in self.execute_child():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   527
            delete(session, subj, self.rtype, obj)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   528
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   529
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   530
class UpdateStep(Step):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   531
    """step consisting in updating entities / adding relations from relations
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   532
    definitions and from results fetched in previous step
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   533
    """
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   534
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   535
    def __init__(self, plan, updatedefs, attributes):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   536
        Step.__init__(self, plan)
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   537
        self.updatedefs = updatedefs
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   538
        self.attributes = attributes
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1016
diff changeset
   539
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   540
    def execute(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   541
        """execute this step"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   542
        session = self.plan.session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   543
        repo = session.repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   544
        edefs = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   545
        # insert relations
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   546
        if self.children:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   547
            result = self.execute_child()
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   548
        else:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   549
            result = [[]]
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   550
        for i, row in enumerate(result):
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   551
            newrow = []
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   552
            for (lhsinfo, rhsinfo, rschema) in self.updatedefs:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   553
                lhsval = _handle_relterm(lhsinfo, row, newrow)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   554
                rhsval = _handle_relterm(rhsinfo, row, newrow)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   555
                if rschema.final or rschema.inlined:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   556
                    eid = typed_eid(lhsval)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   557
                    try:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   558
                        edef = edefs[eid]
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   559
                    except KeyError:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   560
                        edefs[eid] = edef = session.entity_from_eid(eid)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   561
                    edef[str(rschema)] = rhsval
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   562
                else:
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   563
                    repo.glob_add_relation(session, lhsval, str(rschema), rhsval)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   564
            result[i] = newrow
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   565
        # update entities
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   566
        for eid, edef in edefs.iteritems():
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   567
            repo.glob_update_entity(session, edef, self.attributes)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   568
        return result
4764
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   569
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   570
def _handle_relterm(info, row, newrow):
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   571
    if info[0] is _CONSTANT:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   572
        val = info[1]
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   573
    else: # _FROM_SUBSTEP
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   574
        val = row[info[1]]
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   575
    if info[-1] is not None:
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   576
        newrow.append(val)
ec9c20c6b9f7 [repo] improve planning of insert/update queries: do not select affected constants so the don't go and back to/from the source.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   577
    return val