author | Julien Cristau <julien.cristau@logilab.fr> |
Thu, 12 Dec 2013 16:18:58 +0100 | |
changeset 9371 | 1348202527a6 |
parent 8694 | d901c36bcfce |
permissions | -rw-r--r-- |
7521
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
1 |
# copyright 2003-2011 LOGILAB S.A. (Paris, FRANCE), all rights reserved. |
5421
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
2 |
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
3 |
# |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
4 |
# This file is part of CubicWeb. |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
5 |
# |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
6 |
# CubicWeb is free software: you can redistribute it and/or modify it under the |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
7 |
# terms of the GNU Lesser General Public License as published by the Free |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
8 |
# Software Foundation, either version 2.1 of the License, or (at your option) |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
9 |
# any later version. |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
10 |
# |
5424
8ecbcbff9777
replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5421
diff
changeset
|
11 |
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT |
5421
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
12 |
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
13 |
# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
14 |
# details. |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
15 |
# |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
16 |
# You should have received a copy of the GNU Lesser General Public License along |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
17 |
# with CubicWeb. If not, see <http://www.gnu.org/licenses/>. |
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
18 |
"""Defines the diferent querier steps usable in plans. |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
19 |
|
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
20 |
FIXME : this code needs refactoring. Some problems : |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
21 |
* get data from the parent plan, the latest step, temporary table... |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
22 |
* each step has is own members (this is not necessarily bad, but a bit messy |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
23 |
for now) |
5784
8b2264e485bc
[ms] orderby/groupby/having attributes of Select node should be resetted to empty tuple, not None
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
24 |
""" |
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
25 |
__docformat__ = "restructuredtext en" |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
26 |
|
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
27 |
from rql.nodes import VariableRef, Variable, Function |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
28 |
|
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
29 |
from cubicweb.server.ssplanner import (LimitOffsetMixIn, Step, OneFetchStep, |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
30 |
varmap_test_repr, offset_result) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
31 |
|
1802
d628defebc17
delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
470
diff
changeset
|
32 |
AGGR_TRANSFORMS = {'COUNT':'SUM', 'MIN':'MIN', 'MAX':'MAX', 'SUM': 'SUM'} |
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
33 |
|
7521
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
34 |
class remove_and_restore_clauses(object): |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
35 |
def __init__(self, union, keepgroup): |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
36 |
self.union = union |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
37 |
self.keepgroup = keepgroup |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
38 |
self.clauses = None |
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
39 |
|
7521
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
40 |
def __enter__(self): |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
41 |
self.clauses = clauses = [] |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
42 |
for select in self.union.children: |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
43 |
if self.keepgroup: |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
44 |
having, orderby = select.having, select.orderby |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
45 |
select.having, select.orderby = (), () |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
46 |
clauses.append( (having, orderby) ) |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
47 |
else: |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
48 |
groupby, having, orderby = select.groupby, select.having, select.orderby |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
49 |
select.groupby, select.having, select.orderby = (), (), () |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
50 |
clauses.append( (groupby, having, orderby) ) |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
51 |
|
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
52 |
def __exit__(self, exctype, exc, traceback): |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
53 |
for i, select in enumerate(self.union.children): |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
54 |
if self.keepgroup: |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
55 |
select.having, select.orderby = self.clauses[i] |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
56 |
else: |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
57 |
select.groupby, select.having, select.orderby = self.clauses[i] |
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
58 |
|
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
59 |
|
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
60 |
class FetchStep(OneFetchStep): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
61 |
"""step consisting in fetching data from sources, and storing result in |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
62 |
a temporary table |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
63 |
""" |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
64 |
def __init__(self, plan, union, sources, table, keepgroup, inputmap=None): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
65 |
OneFetchStep.__init__(self, plan, union, sources) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
66 |
# temporary table to store step result |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
67 |
self.table = table |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
68 |
# should groupby clause be kept or not |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
69 |
self.keepgroup = keepgroup |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
70 |
# variables mapping to use as input |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
71 |
self.inputmap = inputmap |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
72 |
# output variable mapping |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
73 |
srqlst = union.children[0] # sample select node |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
74 |
# add additional information to the output mapping |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
75 |
self.outputmap = plan.init_temp_table(table, srqlst.selection, |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
76 |
srqlst.solutions[0]) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
77 |
for vref in srqlst.selection: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
78 |
if not isinstance(vref, VariableRef): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
79 |
continue |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
80 |
var = vref.variable |
5004
4cc020ee70e2
le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4212
diff
changeset
|
81 |
if var.stinfo.get('attrvars'): |
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
82 |
for lhsvar, rtype in var.stinfo['attrvars']: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
83 |
if lhsvar.name in srqlst.defined_vars: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
84 |
key = '%s.%s' % (lhsvar.name, rtype) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
85 |
self.outputmap[key] = self.outputmap[var.name] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
86 |
else: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
87 |
rschema = self.plan.schema.rschema |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
88 |
for rel in var.stinfo['rhsrelations']: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
89 |
if rschema(rel.r_type).inlined: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
90 |
lhsvar = rel.children[0] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
91 |
if lhsvar.name in srqlst.defined_vars: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
92 |
key = '%s.%s' % (lhsvar.name, rel.r_type) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
93 |
self.outputmap[key] = self.outputmap[var.name] |
1802
d628defebc17
delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
470
diff
changeset
|
94 |
|
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
95 |
def execute(self): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
96 |
"""execute this step""" |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
97 |
self.execute_children() |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
98 |
plan = self.plan |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
99 |
plan.create_temp_table(self.table) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
100 |
union = self.union |
7521
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
101 |
with remove_and_restore_clauses(union, self.keepgroup): |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
102 |
for source in self.sources: |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
103 |
source.flying_insert(self.table, plan.session, union, plan.args, |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
104 |
self.inputmap) |
1802
d628defebc17
delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
470
diff
changeset
|
105 |
|
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
106 |
def mytest_repr(self): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
107 |
"""return a representation of this step suitable for test""" |
7521
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
108 |
with remove_and_restore_clauses(self.union, self.keepgroup): |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
109 |
try: |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
110 |
inputmap = varmap_test_repr(self.inputmap, self.plan.tablesinorder) |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
111 |
outputmap = varmap_test_repr(self.outputmap, self.plan.tablesinorder) |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
112 |
except AttributeError: |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
113 |
inputmap = self.inputmap |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
114 |
outputmap = self.outputmap |
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
115 |
return (self.__class__.__name__, |
7521
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
116 |
sorted((r.as_string(kwargs=self.plan.args), r.solutions) |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
117 |
for r in self.union.children), |
a5a5eea29c1b
[querier] turn remove_clauses/restore_clauses into a context manager
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7108
diff
changeset
|
118 |
sorted(self.sources), inputmap, outputmap) |
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
119 |
|
1802
d628defebc17
delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
470
diff
changeset
|
120 |
|
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
121 |
class AggrStep(LimitOffsetMixIn, Step): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
122 |
"""step consisting in making aggregat from temporary data in the system |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
123 |
source |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
124 |
""" |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
125 |
def __init__(self, plan, selection, select, table, outputtable=None): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
126 |
Step.__init__(self, plan) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
127 |
# original selection |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
128 |
self.selection = selection |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
129 |
# original Select RQL tree |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
130 |
self.select = select |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
131 |
# table where are located temporary results |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
132 |
self.table = table |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
133 |
# optional table where to write results |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
134 |
self.outputtable = outputtable |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
135 |
if outputtable is not None: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
136 |
plan.init_temp_table(outputtable, selection, select.solutions[0]) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
137 |
|
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
138 |
#self.inputmap = inputmap |
1802
d628defebc17
delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
470
diff
changeset
|
139 |
|
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
140 |
def mytest_repr(self): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
141 |
"""return a representation of this step suitable for test""" |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
142 |
try: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
143 |
# rely on a monkey patch (cf unittest_querier) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
144 |
table = self.plan.tablesinorder[self.table] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
145 |
outputtable = self.outputtable and self.plan.tablesinorder[self.outputtable] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
146 |
except AttributeError: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
147 |
# not monkey patched |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
148 |
table = self.table |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
149 |
outputtable = self.outputtable |
5768
1e73a466aa69
[fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
150 |
sql = self.get_sql().replace(self.table, table) |
1e73a466aa69
[fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
151 |
return (self.__class__.__name__, sql, outputtable) |
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
152 |
|
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
153 |
def execute(self): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
154 |
"""execute this step""" |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
155 |
self.execute_children() |
5768
1e73a466aa69
[fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
156 |
sql = self.get_sql() |
1e73a466aa69
[fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
157 |
if self.outputtable: |
1e73a466aa69
[fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
158 |
self.plan.create_temp_table(self.outputtable) |
1e73a466aa69
[fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
159 |
sql = 'INSERT INTO %s %s' % (self.outputtable, sql) |
8151
8e3fa37cd8aa
[multi-sources] call doexec rather than sqlexec since we do not want to call process_result (as there aren't any and this breaks with lgdb default)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7521
diff
changeset
|
160 |
self.plan.syssource.doexec(self.plan.session, sql, self.plan.args) |
8e3fa37cd8aa
[multi-sources] call doexec rather than sqlexec since we do not want to call process_result (as there aren't any and this breaks with lgdb default)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7521
diff
changeset
|
161 |
else: |
8e3fa37cd8aa
[multi-sources] call doexec rather than sqlexec since we do not want to call process_result (as there aren't any and this breaks with lgdb default)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7521
diff
changeset
|
162 |
return self.plan.sqlexec(sql, self.plan.args) |
5768
1e73a466aa69
[fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
163 |
|
1e73a466aa69
[fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
164 |
def get_sql(self): |
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
165 |
self.inputmap = inputmap = self.children[-1].outputmap |
7108
bcdf22734059
Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5811
diff
changeset
|
166 |
dbhelper=self.plan.syssource.dbhelper |
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
167 |
# get the select clause |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
168 |
clause = [] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
169 |
for i, term in enumerate(self.selection): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
170 |
try: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
171 |
var_name = inputmap[term.as_string()] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
172 |
except KeyError: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
173 |
var_name = 'C%s' % i |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
174 |
if isinstance(term, Function): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
175 |
# we have to translate some aggregat function |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
176 |
# (for instance COUNT -> SUM) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
177 |
orig_name = term.name |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
178 |
try: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
179 |
term.name = AGGR_TRANSFORMS[term.name] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
180 |
# backup and reduce children |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
181 |
orig_children = term.children |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
182 |
term.children = [VariableRef(Variable(var_name))] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
183 |
clause.append(term.accept(self)) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
184 |
# restaure the tree XXX necessary? |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
185 |
term.name = orig_name |
1802
d628defebc17
delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
470
diff
changeset
|
186 |
term.children = orig_children |
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
187 |
except KeyError: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
188 |
clause.append(var_name) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
189 |
else: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
190 |
clause.append(var_name) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
191 |
for vref in term.iget_nodes(VariableRef): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
192 |
inputmap[vref.name] = var_name |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
193 |
# XXX handle distinct with non selected sort term |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
194 |
if self.select.distinct: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
195 |
sql = ['SELECT DISTINCT %s' % ', '.join(clause)] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
196 |
else: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
197 |
sql = ['SELECT %s' % ', '.join(clause)] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
198 |
sql.append("FROM %s" % self.table) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
199 |
# get the group/having clauses |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
200 |
if self.select.groupby: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
201 |
clause = [inputmap[var.name] for var in self.select.groupby] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
202 |
grouped = set(var.name for var in self.select.groupby) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
203 |
sql.append('GROUP BY %s' % ', '.join(clause)) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
204 |
else: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
205 |
grouped = None |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
206 |
if self.select.having: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
207 |
clause = [term.accept(self) for term in self.select.having] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
208 |
sql.append('HAVING %s' % ', '.join(clause)) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
209 |
# get the orderby clause |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
210 |
if self.select.orderby: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
211 |
clause = [] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
212 |
for sortterm in self.select.orderby: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
213 |
sqlterm = sortterm.term.accept(self) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
214 |
if sortterm.asc: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
215 |
clause.append(sqlterm) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
216 |
else: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
217 |
clause.append('%s DESC' % sqlterm) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
218 |
if grouped is not None: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
219 |
for vref in sortterm.iget_nodes(VariableRef): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
220 |
if not vref.name in grouped: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
221 |
sql[-1] += ', ' + self.inputmap[vref.name] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
222 |
grouped.add(vref.name) |
7108
bcdf22734059
Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5811
diff
changeset
|
223 |
sql = dbhelper.sql_add_order_by(' '.join(sql), |
bcdf22734059
Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5811
diff
changeset
|
224 |
clause, |
bcdf22734059
Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5811
diff
changeset
|
225 |
None, False, |
bcdf22734059
Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5811
diff
changeset
|
226 |
self.limit or self.offset) |
bcdf22734059
Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5811
diff
changeset
|
227 |
else: |
bcdf22734059
Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5811
diff
changeset
|
228 |
sql = ' '.join(sql) |
bcdf22734059
Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5811
diff
changeset
|
229 |
clause = None |
bcdf22734059
Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5811
diff
changeset
|
230 |
|
bcdf22734059
Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5811
diff
changeset
|
231 |
sql = dbhelper.sql_add_limit_offset(sql, self.limit, self.offset, clause) |
bcdf22734059
Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5811
diff
changeset
|
232 |
return sql |
1802
d628defebc17
delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
470
diff
changeset
|
233 |
|
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
234 |
def visit_function(self, function): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
235 |
"""generate SQL name for a function""" |
5768
1e73a466aa69
[fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
236 |
try: |
1e73a466aa69
[fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
237 |
return self.children[0].outputmap[str(function)] |
1e73a466aa69
[fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
238 |
except KeyError: |
1e73a466aa69
[fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
239 |
return '%s(%s)' % (function.name, |
1e73a466aa69
[fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
240 |
','.join(c.accept(self) for c in function.children)) |
1802
d628defebc17
delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
470
diff
changeset
|
241 |
|
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
242 |
def visit_variableref(self, variableref): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
243 |
"""get the sql name for a variable reference""" |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
244 |
try: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
245 |
return self.inputmap[variableref.name] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
246 |
except KeyError: # XXX duh? explain |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
247 |
return variableref.variable.name |
1802
d628defebc17
delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
470
diff
changeset
|
248 |
|
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
249 |
def visit_constant(self, constant): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
250 |
"""generate SQL name for a constant""" |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
251 |
assert constant.type == 'Int' |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
252 |
return str(constant.value) |
1802
d628defebc17
delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
470
diff
changeset
|
253 |
|
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
254 |
|
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
255 |
class UnionStep(LimitOffsetMixIn, Step): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
256 |
"""union results of child in-memory steps (e.g. OneFetchStep / AggrStep)""" |
1802
d628defebc17
delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
470
diff
changeset
|
257 |
|
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
258 |
def execute(self): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
259 |
"""execute this step""" |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
260 |
result = [] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
261 |
limit = olimit = self.limit |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
262 |
offset = self.offset |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
263 |
assert offset != 0 |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
264 |
if offset is not None: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
265 |
limit = limit + offset |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
266 |
for step in self.children: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
267 |
if limit is not None: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
268 |
if offset is None: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
269 |
limit = olimit - len(result) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
270 |
step.set_limit_offset(limit, None) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
271 |
result_ = step.execute() |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
272 |
if offset is not None: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
273 |
offset, result_ = offset_result(offset, result_) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
274 |
result += result_ |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
275 |
if limit is not None: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
276 |
if len(result) >= olimit: |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
277 |
return result[:olimit] |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
278 |
return result |
1802
d628defebc17
delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
470
diff
changeset
|
279 |
|
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
280 |
def mytest_repr(self): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
281 |
"""return a representation of this step suitable for test""" |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
282 |
return (self.__class__.__name__, self.limit, self.offset) |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
283 |
|
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
284 |
|
341
0a426be2f3a2
fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
257
diff
changeset
|
285 |
class IntersectStep(UnionStep): |
0a426be2f3a2
fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
257
diff
changeset
|
286 |
"""return intersection of results of child in-memory steps (e.g. OneFetchStep / AggrStep)""" |
1802
d628defebc17
delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
470
diff
changeset
|
287 |
|
341
0a426be2f3a2
fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
257
diff
changeset
|
288 |
def execute(self): |
0a426be2f3a2
fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
257
diff
changeset
|
289 |
"""execute this step""" |
0a426be2f3a2
fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
257
diff
changeset
|
290 |
result = set() |
0a426be2f3a2
fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
257
diff
changeset
|
291 |
for step in self.children: |
0a426be2f3a2
fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
257
diff
changeset
|
292 |
result &= frozenset(step.execute()) |
0a426be2f3a2
fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
257
diff
changeset
|
293 |
result = list(result) |
0a426be2f3a2
fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
257
diff
changeset
|
294 |
if self.offset: |
448 | 295 |
result = result[self.offset:] |
341
0a426be2f3a2
fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
257
diff
changeset
|
296 |
if self.limit: |
470 | 297 |
result = result[:self.limit] |
341
0a426be2f3a2
fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
257
diff
changeset
|
298 |
return result |
0a426be2f3a2
fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
257
diff
changeset
|
299 |
|
0a426be2f3a2
fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
257
diff
changeset
|
300 |
|
257
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
301 |
class UnionFetchStep(Step): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
302 |
"""union results of child steps using temporary tables (e.g. FetchStep)""" |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
303 |
|
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
304 |
def execute(self): |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
305 |
"""execute this step""" |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
306 |
self.execute_children() |
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
307 |
|
4c7d3af7e94d
restore multi-sources capabilities
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
308 |
|
341
0a426be2f3a2
fixes for some cases of neged inlined relations, may have to use (new) IntersectStep. XXX: write IntersectFetchStep
Sylvain Thenault <sylvain.thenault@logilab.fr>
parents:
257
diff
changeset
|
309 |
__all__ = ('FetchStep', 'AggrStep', 'UnionStep', 'UnionFetchStep', 'IntersectStep') |