server/mssteps.py
author Julien Cristau <julien.cristau@logilab.fr>
Thu, 12 Dec 2013 16:18:58 +0100
changeset 9371 1348202527a6
parent 8694 d901c36bcfce
permissions -rw-r--r--
[cwctl] Don't check for old lgc versions Support for check_duplicated_command in CommandLine was added in 2011 for 0.55, and we already depend on 0.59.0.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
7521
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
     1
# copyright 2003-2011 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    18
"""Defines the diferent querier steps usable in plans.
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    19
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    20
FIXME : this code needs refactoring. Some problems :
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    21
* get data from the parent plan, the latest step, temporary table...
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    22
* each step has is own members (this is not necessarily bad, but a bit messy
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    23
  for now)
5784
8b2264e485bc [ms] orderby/groupby/having attributes of Select node should be resetted to empty tuple, not None
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
    24
"""
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    25
__docformat__ = "restructuredtext en"
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    26
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    27
from rql.nodes import VariableRef, Variable, Function
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    28
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    29
from cubicweb.server.ssplanner import (LimitOffsetMixIn, Step, OneFetchStep,
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    30
                                    varmap_test_repr, offset_result)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    31
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 470
diff changeset
    32
AGGR_TRANSFORMS = {'COUNT':'SUM', 'MIN':'MIN', 'MAX':'MAX', 'SUM': 'SUM'}
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    33
7521
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    34
class remove_and_restore_clauses(object):
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    35
    def __init__(self, union, keepgroup):
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    36
        self.union = union
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    37
        self.keepgroup = keepgroup
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    38
        self.clauses = None
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    39
7521
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    40
    def __enter__(self):
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    41
        self.clauses = clauses = []
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    42
        for select in self.union.children:
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    43
            if self.keepgroup:
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    44
                having, orderby = select.having, select.orderby
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    45
                select.having, select.orderby = (), ()
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    46
                clauses.append( (having, orderby) )
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    47
            else:
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    48
                groupby, having, orderby = select.groupby, select.having, select.orderby
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    49
                select.groupby, select.having, select.orderby = (), (), ()
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    50
                clauses.append( (groupby, having, orderby) )
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    51
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    52
    def __exit__(self, exctype, exc, traceback):
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    53
        for i, select in enumerate(self.union.children):
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    54
            if self.keepgroup:
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    55
                select.having, select.orderby = self.clauses[i]
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    56
            else:
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    57
                select.groupby, select.having, select.orderby = self.clauses[i]
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    58
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    59
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    60
class FetchStep(OneFetchStep):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    61
    """step consisting in fetching data from sources, and storing result in
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    62
    a temporary table
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    63
    """
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    64
    def __init__(self, plan, union, sources, table, keepgroup, inputmap=None):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    65
        OneFetchStep.__init__(self, plan, union, sources)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    66
        # temporary table to store step result
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    67
        self.table = table
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    68
        # should groupby clause be kept or not
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    69
        self.keepgroup = keepgroup
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    70
        # variables mapping to use as input
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    71
        self.inputmap = inputmap
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    72
        # output variable mapping
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    73
        srqlst = union.children[0] # sample select node
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    74
        # add additional information to the output mapping
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    75
        self.outputmap = plan.init_temp_table(table, srqlst.selection,
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    76
                                              srqlst.solutions[0])
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    77
        for vref in srqlst.selection:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    78
            if not isinstance(vref, VariableRef):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    79
                continue
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    80
            var = vref.variable
5004
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
    81
            if var.stinfo.get('attrvars'):
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    82
                for lhsvar, rtype in var.stinfo['attrvars']:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    83
                    if lhsvar.name in srqlst.defined_vars:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    84
                        key = '%s.%s' % (lhsvar.name, rtype)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    85
                        self.outputmap[key] = self.outputmap[var.name]
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    86
            else:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    87
                rschema = self.plan.schema.rschema
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    88
                for rel in var.stinfo['rhsrelations']:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    89
                    if rschema(rel.r_type).inlined:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    90
                        lhsvar = rel.children[0]
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    91
                        if lhsvar.name in srqlst.defined_vars:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    92
                            key = '%s.%s' % (lhsvar.name, rel.r_type)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    93
                            self.outputmap[key] = self.outputmap[var.name]
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 470
diff changeset
    94
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    95
    def execute(self):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    96
        """execute this step"""
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    97
        self.execute_children()
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    98
        plan = self.plan
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    99
        plan.create_temp_table(self.table)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   100
        union = self.union
7521
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   101
        with remove_and_restore_clauses(union, self.keepgroup):
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   102
            for source in self.sources:
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   103
                source.flying_insert(self.table, plan.session, union, plan.args,
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   104
                                     self.inputmap)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 470
diff changeset
   105
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   106
    def mytest_repr(self):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   107
        """return a representation of this step suitable for test"""
7521
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   108
        with remove_and_restore_clauses(self.union, self.keepgroup):
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   109
            try:
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   110
                inputmap = varmap_test_repr(self.inputmap, self.plan.tablesinorder)
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   111
                outputmap = varmap_test_repr(self.outputmap, self.plan.tablesinorder)
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   112
            except AttributeError:
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   113
                inputmap = self.inputmap
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   114
                outputmap = self.outputmap
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   115
            return (self.__class__.__name__,
7521
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   116
                    sorted((r.as_string(kwargs=self.plan.args), r.solutions)
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   117
                           for r in self.union.children),
a5a5eea29c1b [querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   118
                    sorted(self.sources), inputmap, outputmap)
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   119
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 470
diff changeset
   120
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   121
class AggrStep(LimitOffsetMixIn, Step):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   122
    """step consisting in making aggregat from temporary data in the system
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   123
    source
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   124
    """
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   125
    def __init__(self, plan, selection, select, table, outputtable=None):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   126
        Step.__init__(self, plan)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   127
        # original selection
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   128
        self.selection = selection
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   129
        # original Select RQL tree
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   130
        self.select = select
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   131
        # table where are located temporary results
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   132
        self.table = table
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   133
        # optional table where to write results
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   134
        self.outputtable = outputtable
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   135
        if outputtable is not None:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   136
            plan.init_temp_table(outputtable, selection, select.solutions[0])
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   137
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   138
        #self.inputmap = inputmap
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 470
diff changeset
   139
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   140
    def mytest_repr(self):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   141
        """return a representation of this step suitable for test"""
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   142
        try:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   143
            # rely on a monkey patch (cf unittest_querier)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   144
            table = self.plan.tablesinorder[self.table]
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   145
            outputtable = self.outputtable and self.plan.tablesinorder[self.outputtable]
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   146
        except AttributeError:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   147
            # not monkey patched
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   148
            table = self.table
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   149
            outputtable = self.outputtable
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   150
        sql = self.get_sql().replace(self.table, table)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   151
        return (self.__class__.__name__, sql, outputtable)
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   152
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   153
    def execute(self):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   154
        """execute this step"""
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   155
        self.execute_children()
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   156
        sql = self.get_sql()
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   157
        if self.outputtable:
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   158
            self.plan.create_temp_table(self.outputtable)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   159
            sql = 'INSERT INTO %s %s' % (self.outputtable, sql)
8151
8e3fa37cd8aa [multi-sources] call doexec rather than sqlexec since we do not want to call process_result (as there aren't any and this breaks with lgdb default)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7521
diff changeset
   160
            self.plan.syssource.doexec(self.plan.session, sql, self.plan.args)
8e3fa37cd8aa [multi-sources] call doexec rather than sqlexec since we do not want to call process_result (as there aren't any and this breaks with lgdb default)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7521
diff changeset
   161
        else:
8e3fa37cd8aa [multi-sources] call doexec rather than sqlexec since we do not want to call process_result (as there aren't any and this breaks with lgdb default)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7521
diff changeset
   162
            return self.plan.sqlexec(sql, self.plan.args)
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   163
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   164
    def get_sql(self):
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   165
        self.inputmap = inputmap = self.children[-1].outputmap
7108
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5811
diff changeset
   166
        dbhelper=self.plan.syssource.dbhelper
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   167
        # get the select clause
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   168
        clause = []
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   169
        for i, term in enumerate(self.selection):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   170
            try:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   171
                var_name = inputmap[term.as_string()]
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   172
            except KeyError:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   173
                var_name = 'C%s' % i
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   174
            if isinstance(term, Function):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   175
                # we have to translate some aggregat function
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   176
                # (for instance COUNT -> SUM)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   177
                orig_name = term.name
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   178
                try:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   179
                    term.name = AGGR_TRANSFORMS[term.name]
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   180
                    # backup and reduce children
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   181
                    orig_children = term.children
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   182
                    term.children = [VariableRef(Variable(var_name))]
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   183
                    clause.append(term.accept(self))
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   184
                    # restaure the tree XXX necessary?
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   185
                    term.name = orig_name
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 470
diff changeset
   186
                    term.children = orig_children
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   187
                except KeyError:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   188
                    clause.append(var_name)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   189
            else:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   190
                clause.append(var_name)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   191
                for vref in term.iget_nodes(VariableRef):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   192
                    inputmap[vref.name] = var_name
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   193
        # XXX handle distinct with non selected sort term
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   194
        if self.select.distinct:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   195
            sql = ['SELECT DISTINCT %s' % ', '.join(clause)]
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   196
        else:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   197
            sql = ['SELECT %s' % ', '.join(clause)]
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   198
        sql.append("FROM %s" % self.table)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   199
        # get the group/having clauses
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   200
        if self.select.groupby:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   201
            clause = [inputmap[var.name] for var in self.select.groupby]
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   202
            grouped = set(var.name for var in self.select.groupby)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   203
            sql.append('GROUP BY %s' % ', '.join(clause))
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   204
        else:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   205
            grouped = None
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   206
        if self.select.having:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   207
            clause = [term.accept(self) for term in self.select.having]
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   208
            sql.append('HAVING %s' % ', '.join(clause))
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   209
        # get the orderby clause
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   210
        if self.select.orderby:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   211
            clause = []
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   212
            for sortterm in self.select.orderby:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   213
                sqlterm = sortterm.term.accept(self)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   214
                if sortterm.asc:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   215
                    clause.append(sqlterm)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   216
                else:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   217
                    clause.append('%s DESC' % sqlterm)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   218
                if grouped is not None:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   219
                    for vref in sortterm.iget_nodes(VariableRef):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   220
                        if not vref.name in grouped:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   221
                            sql[-1] += ', ' + self.inputmap[vref.name]
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   222
                            grouped.add(vref.name)
7108
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5811
diff changeset
   223
            sql = dbhelper.sql_add_order_by(' '.join(sql),
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5811
diff changeset
   224
                                            clause,
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5811
diff changeset
   225
                                            None, False,
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5811
diff changeset
   226
                                            self.limit or self.offset)
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5811
diff changeset
   227
        else:
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5811
diff changeset
   228
            sql = ' '.join(sql)
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5811
diff changeset
   229
            clause = None
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5811
diff changeset
   230
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5811
diff changeset
   231
        sql = dbhelper.sql_add_limit_offset(sql, self.limit, self.offset, clause)
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5811
diff changeset
   232
        return sql
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 470
diff changeset
   233
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   234
    def visit_function(self, function):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   235
        """generate SQL name for a function"""
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   236
        try:
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   237
            return self.children[0].outputmap[str(function)]
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   238
        except KeyError:
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   239
            return '%s(%s)' % (function.name,
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   240
                               ','.join(c.accept(self) for c in function.children))
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 470
diff changeset
   241
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   242
    def visit_variableref(self, variableref):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   243
        """get the sql name for a variable reference"""
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   244
        try:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   245
            return self.inputmap[variableref.name]
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   246
        except KeyError: # XXX duh? explain
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   247
            return variableref.variable.name
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 470
diff changeset
   248
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   249
    def visit_constant(self, constant):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   250
        """generate SQL name for a constant"""
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   251
        assert constant.type == 'Int'
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   252
        return str(constant.value)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 470
diff changeset
   253
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   254
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   255
class UnionStep(LimitOffsetMixIn, Step):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   256
    """union results of child in-memory steps (e.g. OneFetchStep / AggrStep)"""
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 470
diff changeset
   257
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   258
    def execute(self):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   259
        """execute this step"""
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   260
        result = []
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   261
        limit = olimit = self.limit
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   262
        offset = self.offset
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   263
        assert offset != 0
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   264
        if offset is not None:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   265
            limit = limit + offset
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   266
        for step in self.children:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   267
            if limit is not None:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   268
                if offset is None:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   269
                    limit = olimit - len(result)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   270
                step.set_limit_offset(limit, None)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   271
            result_ = step.execute()
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   272
            if offset is not None:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   273
                offset, result_ = offset_result(offset, result_)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   274
            result += result_
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   275
            if limit is not None:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   276
                if len(result) >= olimit:
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   277
                    return result[:olimit]
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   278
        return result
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 470
diff changeset
   279
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   280
    def mytest_repr(self):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   281
        """return a representation of this step suitable for test"""
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   282
        return (self.__class__.__name__, self.limit, self.offset)
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   283
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   284
341
0a426be2f3a2 fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 257
diff changeset
   285
class IntersectStep(UnionStep):
0a426be2f3a2 fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 257
diff changeset
   286
    """return intersection of results of child in-memory steps (e.g. OneFetchStep / AggrStep)"""
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 470
diff changeset
   287
341
0a426be2f3a2 fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 257
diff changeset
   288
    def execute(self):
0a426be2f3a2 fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 257
diff changeset
   289
        """execute this step"""
0a426be2f3a2 fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 257
diff changeset
   290
        result = set()
0a426be2f3a2 fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 257
diff changeset
   291
        for step in self.children:
0a426be2f3a2 fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 257
diff changeset
   292
            result &= frozenset(step.execute())
0a426be2f3a2 fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 257
diff changeset
   293
        result = list(result)
0a426be2f3a2 fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 257
diff changeset
   294
        if self.offset:
448
ce91d0f23ed2 pylint fixes
sylvain.thenault@logilab.fr
parents: 341
diff changeset
   295
            result = result[self.offset:]
341
0a426be2f3a2 fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 257
diff changeset
   296
        if self.limit:
470
sylvain.thenault@logilab.fr
parents: 449
diff changeset
   297
            result = result[:self.limit]
341
0a426be2f3a2 fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 257
diff changeset
   298
        return result
0a426be2f3a2 fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 257
diff changeset
   299
0a426be2f3a2 fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 257
diff changeset
   300
257
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   301
class UnionFetchStep(Step):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   302
    """union results of child steps using temporary tables (e.g. FetchStep)"""
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   303
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   304
    def execute(self):
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   305
        """execute this step"""
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   306
        self.execute_children()
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   307
4c7d3af7e94d restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   308
341
0a426be2f3a2 fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents: 257
diff changeset
   309
__all__ = ('FetchStep', 'AggrStep', 'UnionStep', 'UnionFetchStep', 'IntersectStep')