server/sources/rql2sql.py
author Julien Cristau <julien.cristau@logilab.fr>
Mon, 17 Mar 2014 12:13:14 +0100
changeset 9577 c52441e4a3d7
parent 9543 39f981482e34
child 10450 c5c9e5b6fde0
child 10475 a1e8dbb7215b
permissions -rw-r--r--
[serverctl] use repoapi for db-check, add-source, rebuild-fti commands One more step towards getting rid of dbapi.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8894
diff changeset
     1
# copyright 2003-2013 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5280
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5280
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5280
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5280
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5280
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5280
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5280
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5280
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5280
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5280
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5280
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5280
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5280
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5280
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5280
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
"""RQL to SQL generator for native sources.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
SQL queries optimization
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
~~~~~~~~~~~~~~~~~~~~~~~~
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
    23
1. CWUser X WHERE X in_group G, G name 'users':
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
    25
   CWUser is the only subject entity type for the in_group relation,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
   which allow us to do ::
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
    28
     SELECT eid_from FROM in_group, CWGroup
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
    29
     WHERE in_group.eid_to = CWGroup.eid_from
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
    30
     AND CWGroup.name = 'users'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    31
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    32
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    33
2. Any X WHERE X nonfinal1 Y, Y nonfinal2 Z
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    34
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    35
   -> direct join between nonfinal1 and nonfinal2, whatever X,Y, Z (unless
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    36
      inlined...)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
    37
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    38
      NOT IMPLEMENTED (and quite hard to implement)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    39
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    40
Potential optimization information is collected by the querier, sql generation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    41
is done according to this information
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    42
5280
7e13bb484a19 added note about interesting page about SQL implementation differences
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5013
diff changeset
    43
cross RDMS note : read `Comparison of different SQL implementations`_
7e13bb484a19 added note about interesting page about SQL implementation differences
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5013
diff changeset
    44
by Troels Arvin. Features SQL ISO Standard, PG, mysql, Oracle, MS SQL, DB2
7e13bb484a19 added note about interesting page about SQL implementation differences
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5013
diff changeset
    45
and Informix.
7e13bb484a19 added note about interesting page about SQL implementation differences
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5013
diff changeset
    46
5582
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
    47
.. _Comparison of different SQL implementations: http://www.troels.arvin.dk/db/rdbms
5782
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
    48
"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    49
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    51
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
import threading
7166
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    53
from datetime import datetime, time
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    54
7166
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
    55
from logilab.common.date import utcdatetime, utctime
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
    56
from logilab.database import FunctionDescr, SQL_FUNCTIONS_REGISTRY
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
    57
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    58
from rql import BadRQLQuery, CoercionError
7672
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
    59
from rql.utils import common_parent
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    60
from rql.stmts import Union, Select
9468
39b7a91a3f4c [repo] pylint cleanup, mainly of imports, with a bit of style
Julien Cristau <julien.cristau@logilab.fr>
parents: 9361
diff changeset
    61
from rql.nodes import (VariableRef, Constant, Function, Variable, Or,
39b7a91a3f4c [repo] pylint cleanup, mainly of imports, with a bit of style
Julien Cristau <julien.cristau@logilab.fr>
parents: 9361
diff changeset
    62
                       Not, Comparison, ColumnAlias, Relation, SubQuery)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
    64
from cubicweb import QueryError
9167
c05652b108ce [rql rewrite] move some code from querier to rqlrewrite where it makes more sense.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8894
diff changeset
    65
from cubicweb.rqlrewrite import cleanup_solutions
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1124
diff changeset
    66
from cubicweb.server.sqlutils import SQL_PREFIX
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    67
4794
81075b09ebf1 [rql2sql] fix potential crash when testing _q_invariant on a ColumnAlias instead of a Variable by always considering _q_invariant to false for them
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
    68
ColumnAlias._q_invariant = False # avoid to check for ColumnAlias / Variable
81075b09ebf1 [rql2sql] fix potential crash when testing _q_invariant on a ColumnAlias instead of a Variable by always considering _q_invariant to false for them
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
    69
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
    70
FunctionDescr.source_execute = None
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
    71
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
    72
def default_update_cb_stack(self, stack):
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
    73
    stack.append(self.source_execute)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
    74
FunctionDescr.update_cb_stack = default_update_cb_stack
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
    75
7421
5115daeeb2ff [rql2sql] closes #1697862: ORDERBY SUM(C) cancel GROUPBY by wrongly adding C to grouped variables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7396
diff changeset
    76
get_func_descr = SQL_FUNCTIONS_REGISTRY.get_function
5115daeeb2ff [rql2sql] closes #1697862: ORDERBY SUM(C) cancel GROUPBY by wrongly adding C to grouped variables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7396
diff changeset
    77
5115daeeb2ff [rql2sql] closes #1697862: ORDERBY SUM(C) cancel GROUPBY by wrongly adding C to grouped variables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7396
diff changeset
    78
LENGTH = get_func_descr('LENGTH')
5630
40d7c7e180f1 [storage] source's callback has a new prototype, update usage
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5593
diff changeset
    79
def length_source_execute(source, session, value):
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
    80
    return len(value.getvalue())
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
    81
LENGTH.source_execute = length_source_execute
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
    82
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
    83
def _new_var(select, varname):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    84
    newvar = select.get_variable(varname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    85
    if not 'relations' in newvar.stinfo:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
        # not yet initialized
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
        newvar.prepare_annotation()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    88
        newvar.stinfo['scope'] = select
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    89
        newvar._q_invariant = False
7252
adb1673efa5f [sql generation] fix crash on outer join + ambiguous inlined relation + attribute selection
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7194
diff changeset
    90
        select.selection.append(VariableRef(newvar))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
    return newvar
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
def _fill_to_wrap_rel(var, newselect, towrap, schema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    94
    for rel in var.stinfo['relations'] - var.stinfo['rhsrelations']:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
        rschema = schema.rschema(rel.r_type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
        if rschema.inlined:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    97
            towrap.add( (var, rel) )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    98
            for vref in rel.children[1].iget_nodes(VariableRef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    99
                newivar = _new_var(newselect, vref.name)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   100
                _fill_to_wrap_rel(vref.variable, newselect, towrap, schema)
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3245
diff changeset
   101
        elif rschema.final:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   102
            towrap.add( (var, rel) )
7252
adb1673efa5f [sql generation] fix crash on outer join + ambiguous inlined relation + attribute selection
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7194
diff changeset
   103
            for vref in rel.children[1].iget_nodes(VariableRef):
adb1673efa5f [sql generation] fix crash on outer join + ambiguous inlined relation + attribute selection
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7194
diff changeset
   104
                newivar = _new_var(newselect, vref.name)
adb1673efa5f [sql generation] fix crash on outer join + ambiguous inlined relation + attribute selection
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7194
diff changeset
   105
                newivar.stinfo['attrvar'] = (var, rel.r_type)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
   106
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   107
def rewrite_unstable_outer_join(select, solutions, unstable, schema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   108
    """if some optional variables are unstable, they should be selected in a
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   109
    subquery. This function check this and rewrite the rql syntax tree if
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   110
    necessary (in place). Return a boolean telling if the tree has been modified
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   111
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   112
    modified = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   113
    for varname in tuple(unstable):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   114
        var = select.defined_vars[varname]
5004
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
   115
        if not var.stinfo.get('optrelations'):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   116
            continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   117
        unstable.remove(varname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   118
        newselect = Select()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   119
        myunion = Union()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   120
        myunion.append(newselect)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   121
        # extract aliases / selection
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   122
        newvar = _new_var(newselect, var.name)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   123
        newselect.selection = [VariableRef(newvar)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   124
        towrap_rels = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   125
        _fill_to_wrap_rel(var, newselect, towrap_rels, schema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   126
        # extract relations
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   127
        for var, rel in towrap_rels:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   128
            newrel = rel.copy(newselect)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
            newselect.add_restriction(newrel)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   130
            select.remove_node(rel)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   131
            var.stinfo['relations'].remove(rel)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   132
            newvar.stinfo['relations'].add(newrel)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   133
            if rel.optional in ('left', 'both'):
5004
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
   134
                newvar.add_optional_relation(newrel)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
            for vref in newrel.children[1].iget_nodes(VariableRef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
                var = vref.variable
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
                var.stinfo['relations'].add(newrel)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   138
                var.stinfo['rhsrelations'].add(newrel)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   139
                if rel.optional in ('right', 'both'):
5004
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
   140
                    var.add_optional_relation(newrel)
6050
20af96a6fffc [rql2sql] add test for rql bug #37423, which triggered a cw bug fixed by this changeset (infinite loop with some queries...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6003
diff changeset
   141
        if not select.where and not modified:
20af96a6fffc [rql2sql] add test for rql bug #37423, which triggered a cw bug fixed by this changeset (infinite loop with some queries...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6003
diff changeset
   142
            # oops, generated the same thing as the original select....
20af96a6fffc [rql2sql] add test for rql bug #37423, which triggered a cw bug fixed by this changeset (infinite loop with some queries...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6003
diff changeset
   143
            # restore original query, else we'll indefinitly loop
20af96a6fffc [rql2sql] add test for rql bug #37423, which triggered a cw bug fixed by this changeset (infinite loop with some queries...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6003
diff changeset
   144
            for var, rel in towrap_rels:
20af96a6fffc [rql2sql] add test for rql bug #37423, which triggered a cw bug fixed by this changeset (infinite loop with some queries...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6003
diff changeset
   145
                select.add_restriction(rel)
20af96a6fffc [rql2sql] add test for rql bug #37423, which triggered a cw bug fixed by this changeset (infinite loop with some queries...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6003
diff changeset
   146
            continue
20af96a6fffc [rql2sql] add test for rql bug #37423, which triggered a cw bug fixed by this changeset (infinite loop with some queries...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6003
diff changeset
   147
        modified = True
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   148
        # extract subquery solutions
3736
07196bda2456 take care to already existant solutions local variable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   149
        mysolutions = [sol.copy() for sol in solutions]
07196bda2456 take care to already existant solutions local variable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   150
        cleanup_solutions(newselect, mysolutions)
6050
20af96a6fffc [rql2sql] add test for rql bug #37423, which triggered a cw bug fixed by this changeset (infinite loop with some queries...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6003
diff changeset
   151
        newselect.set_possible_types(mysolutions)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   152
        # full sub-query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   153
        aliases = [VariableRef(select.get_variable(avar.name, i))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   154
                   for i, avar in enumerate(newselect.selection)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   155
        select.add_subquery(SubQuery(aliases, myunion), check=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   156
    return modified
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   157
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   158
def _new_solutions(rqlst, solutions):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   159
    """first filter out subqueries variables from solutions"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
    newsolutions = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   161
    for origsol in solutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   162
        asol = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   163
        for vname in rqlst.defined_vars:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   164
            asol[vname] = origsol[vname]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   165
        if not asol in newsolutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   166
            newsolutions.append(asol)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   167
    return newsolutions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   168
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   169
def remove_unused_solutions(rqlst, solutions, varmap, schema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   170
    """cleanup solutions: remove solutions where invariant variables are taking
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   171
    different types
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   172
    """
3852
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   173
    newsols = _new_solutions(rqlst, solutions)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   174
    existssols = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   175
    unstable = set()
3852
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   176
    invariants = {}
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   177
    for vname, var in rqlst.defined_vars.iteritems():
3852
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   178
        vtype = newsols[0][vname]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   179
        if var._q_invariant or vname in varmap:
3852
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   180
            # remove invariant variable from solutions to remove duplicates
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   181
            # later, then reinserting a type for the variable even later
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   182
            for sol in newsols:
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   183
                invariants.setdefault(id(sol), {})[vname] = sol.pop(vname)
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   184
        elif var.scope is not rqlst:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   185
            # move appart variables which are in a EXISTS scope and are variating
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   186
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   187
                thisexistssols, thisexistsvars = existssols[var.scope]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   188
            except KeyError:
3852
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   189
                thisexistssols = [newsols[0]]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   190
                thisexistsvars = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   191
                existssols[var.scope] = thisexistssols, thisexistsvars
3852
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   192
            for i in xrange(len(newsols)-1, 0, -1):
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   193
                if vtype != newsols[i][vname]:
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   194
                    thisexistssols.append(newsols.pop(i))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   195
                    thisexistsvars.add(vname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   196
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   197
            # remember unstable variables
3852
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   198
            for i in xrange(1, len(newsols)):
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   199
                if vtype != newsols[i][vname]:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   200
                    unstable.add(vname)
3852
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   201
    if invariants:
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   202
        # filter out duplicates
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   203
        newsols_ = []
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   204
        for sol in newsols:
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   205
            if not sol in newsols_:
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   206
                newsols_.append(sol)
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   207
        newsols = newsols_
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   208
        # reinsert solutions for invariants
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   209
        for sol in newsols:
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   210
            for invvar, vartype in invariants[id(sol)].iteritems():
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   211
                sol[invvar] = vartype
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   212
        for sol in existssols:
3866
2783c166ad1a fix potential key error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3852
diff changeset
   213
            try:
2783c166ad1a fix potential key error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3852
diff changeset
   214
                for invvar, vartype in invariants[id(sol)].iteritems():
2783c166ad1a fix potential key error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3852
diff changeset
   215
                    sol[invvar] = vartype
2783c166ad1a fix potential key error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3852
diff changeset
   216
            except KeyError:
2783c166ad1a fix potential key error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3852
diff changeset
   217
                continue
3852
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   218
    if len(newsols) > 1:
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   219
        if rewrite_unstable_outer_join(rqlst, newsols, unstable, schema):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   220
            # remove variables extracted to subqueries from solutions
3852
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   221
            newsols = _new_solutions(rqlst, newsols)
03121ca1f85e test and fix case where remove_unsused_solutions remove some solutions that should be kept
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3815
diff changeset
   222
    return newsols, existssols, unstable
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   223
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   224
def relation_info(relation):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   225
    lhs, rhs = relation.get_variable_parts()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   226
    try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   227
        lhs = lhs.variable
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   228
        lhsconst = lhs.stinfo['constnode']
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   229
    except AttributeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   230
        lhsconst = lhs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   231
        lhs = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   232
    except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   233
        lhsconst = None # ColumnAlias
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   234
    try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   235
        rhs = rhs.variable
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   236
        rhsconst = rhs.stinfo['constnode']
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   237
    except AttributeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   238
        rhsconst = rhs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   239
        rhs = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   240
    except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   241
        rhsconst = None # ColumnAlias
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   242
    return lhs, lhsconst, rhs, rhsconst
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   243
7579
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   244
def sort_term_selection(sorts, rqlst, groups):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   245
    # XXX beurk
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   246
    if isinstance(rqlst, list):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   247
        def append(term):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   248
            rqlst.append(term)
7579
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   249
        selectionidx = set(str(term) for term in rqlst)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   250
    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   251
        def append(term):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   252
            rqlst.selection.append(term.copy(rqlst))
7579
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   253
        selectionidx = set(str(term) for term in rqlst.selection)
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   254
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   255
    for sortterm in sorts:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   256
        term = sortterm.term
7579
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   257
        if not isinstance(term, Constant) and not str(term) in selectionidx:
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   258
            selectionidx.add(str(term))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   259
            append(term)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   260
            if groups:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   261
                for vref in term.iget_nodes(VariableRef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   262
                    if not vref in groups:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   263
                        groups.append(vref)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
   264
7579
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   265
def fix_selection_and_group(rqlst, needwrap, selectsortterms,
3752
4c77a1653374 when a query is grouped, ensure sort terms are grouped as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3736
diff changeset
   266
                            sorts, groups, having):
4c77a1653374 when a query is grouped, ensure sort terms are grouped as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3736
diff changeset
   267
    if selectsortterms and sorts:
7579
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   268
        sort_term_selection(sorts, rqlst, not needwrap and groups)
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   269
    groupvrefs = [vref for term in groups for vref in term.iget_nodes(VariableRef)]
3752
4c77a1653374 when a query is grouped, ensure sort terms are grouped as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3736
diff changeset
   270
    if sorts and groups:
3754
a9c641f6193b document
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3752
diff changeset
   271
        # when a query is grouped, ensure sort terms are grouped as well
3752
4c77a1653374 when a query is grouped, ensure sort terms are grouped as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3736
diff changeset
   272
        for sortterm in sorts:
4c77a1653374 when a query is grouped, ensure sort terms are grouped as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3736
diff changeset
   273
            term = sortterm.term
7421
5115daeeb2ff [rql2sql] closes #1697862: ORDERBY SUM(C) cancel GROUPBY by wrongly adding C to grouped variables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7396
diff changeset
   274
            if not (isinstance(term, Constant) or \
5115daeeb2ff [rql2sql] closes #1697862: ORDERBY SUM(C) cancel GROUPBY by wrongly adding C to grouped variables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7396
diff changeset
   275
                    (isinstance(term, Function) and
5115daeeb2ff [rql2sql] closes #1697862: ORDERBY SUM(C) cancel GROUPBY by wrongly adding C to grouped variables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7396
diff changeset
   276
                     get_func_descr(term.name).aggregat)):
3752
4c77a1653374 when a query is grouped, ensure sort terms are grouped as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3736
diff changeset
   277
                for vref in term.iget_nodes(VariableRef):
7579
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   278
                    if not vref in groupvrefs:
3752
4c77a1653374 when a query is grouped, ensure sort terms are grouped as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3736
diff changeset
   279
                        groups.append(vref)
7579
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   280
                        groupvrefs.append(vref)
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   281
    if needwrap and (groups or having):
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   282
        selectedidx = set(vref.name for term in rqlst.selection
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   283
                          for vref in term.get_nodes(VariableRef))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   284
        if groups:
7579
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   285
            for vref in groupvrefs:
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   286
                if vref.name not in selectedidx:
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   287
                    selectedidx.add(vref.name)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   288
                    rqlst.selection.append(vref)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   289
        if having:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
            for term in having:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   291
                for vref in term.iget_nodes(VariableRef):
7579
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   292
                    if vref.name not in selectedidx:
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   293
                        selectedidx.add(vref.name)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   294
                        rqlst.selection.append(vref)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   295
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   296
def iter_mapped_var_sels(stmt, variable):
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   297
    # variable is a Variable or ColumnAlias node mapped to a source side
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   298
    # callback
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   299
    if not (len(variable.stinfo['rhsrelations']) <= 1 and # < 1 on column alias
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   300
            variable.stinfo['selected']):
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   301
        raise QueryError("can't use %s as a restriction variable"
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   302
                         % variable.name)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   303
    for selectidx in variable.stinfo['selected']:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   304
        vrefs = stmt.selection[selectidx].get_nodes(VariableRef)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   305
        if len(vrefs) != 1:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   306
            raise QueryError()
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   307
        yield selectidx, vrefs[0]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   308
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   309
def update_source_cb_stack(state, stmt, node, stack):
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   310
    while True:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   311
        node = node.parent
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   312
        if node is stmt:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   313
            break
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   314
        if not isinstance(node, Function):
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   315
            raise QueryError()
7421
5115daeeb2ff [rql2sql] closes #1697862: ORDERBY SUM(C) cancel GROUPBY by wrongly adding C to grouped variables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7396
diff changeset
   316
        funcd = get_func_descr(node.name)
5115daeeb2ff [rql2sql] closes #1697862: ORDERBY SUM(C) cancel GROUPBY by wrongly adding C to grouped variables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7396
diff changeset
   317
        if funcd.source_execute is None:
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   318
            raise QueryError('%s can not be called on mapped attribute'
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   319
                             % node.name)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   320
        state.source_cb_funcs.add(node)
7421
5115daeeb2ff [rql2sql] closes #1697862: ORDERBY SUM(C) cancel GROUPBY by wrongly adding C to grouped variables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7396
diff changeset
   321
        funcd.update_cb_stack(stack)
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   322
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   323
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   324
# IGenerator implementation for RQL->SQL #######################################
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
class StateInfo(object):
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   327
    """this class stores data accumulated during the RQL syntax tree visit
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   328
    for later SQL generation.
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   329
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   330
    Attributes related to OUTER JOIN handling
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   331
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   332
    * `outer_chains`, list of list of strings. Each list represent a tables
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   333
      that have to be outer joined together.
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   334
8238
087bb529035c [spelling] fix dictionnary -> dictionary typo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7902
diff changeset
   335
    * `outer_tables`, dictionary used as index of tables used in outer join ::
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   336
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   337
        'table alias': (outertype, [conditions], [chain])
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   338
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   339
      where:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   340
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   341
      * `outertype` is one of None, 'LEFT', 'RIGHT', 'FULL'
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   342
      * `conditions` is a list of join conditions (string)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   343
      * `chain` is a list of table alias (the *outer chain*) in which the key
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   344
        alias appears
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   345
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   346
    * `outer_pending` is a dictionary containing some conditions that will have
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   347
      to be added to the outer join when the table will be turned into an
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   348
      outerjoin ::
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   349
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   350
       'table alias': [conditions]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   351
    """
5582
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   352
    def __init__(self, select, existssols, unstablevars):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   353
        self.existssols = existssols
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   354
        self.unstablevars = unstablevars
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   355
        self.subtables = {}
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   356
        self.needs_source_cb = None
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   357
        self.subquery_source_cb = None
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   358
        self.source_cb_funcs = set()
5582
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   359
        self.scopes = {select: 0}
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   360
        self.scope_nodes = []
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
   361
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   362
    def reset(self, solution):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   363
        """reset some visit variables"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   364
        self.solution = solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   365
        self.count = 0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   366
        self.done = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   367
        self.tables = self.subtables.copy()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   368
        self.actual_tables = [[]]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   369
        for _, tsql in self.tables.itervalues():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   370
            self.actual_tables[-1].append(tsql)
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   371
        self.outer_chains = []
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   372
        self.outer_tables = {}
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   373
        self.outer_pending = {}
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   374
        self.duplicate_switches = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   375
        self.aliases = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   376
        self.restrictions = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   377
        self._restr_stack = []
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
   378
        self.ignore_varmap = False
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   379
        self._needs_source_cb = {}
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   380
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   381
    def merge_source_cbs(self, needs_source_cb):
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   382
        if self.needs_source_cb is None:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   383
            self.needs_source_cb = needs_source_cb
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   384
        elif needs_source_cb != self.needs_source_cb:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   385
            raise QueryError('query fetch some source mapped attribute, some not')
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   386
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   387
    def finalize_source_cbs(self):
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   388
        if self.subquery_source_cb is not None:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   389
            self.needs_source_cb.update(self.subquery_source_cb)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
   390
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   391
    def add_restriction(self, restr):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   392
        if restr:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   393
            self.restrictions.append(restr)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
   394
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   395
    def iter_exists_sols(self, exists):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   396
        if not exists in self.existssols:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   397
            yield 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   398
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   399
        thisexistssols, thisexistsvars = self.existssols[exists]
7042
8bcf33c493f8 [sql generation] fix exists table scope pb: when iterating other solutions inner to an EXISTS subquery, we should reset variables which have this exists node as scope
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6940
diff changeset
   400
        # when iterating other solutions inner to an EXISTS subquery, we should
8bcf33c493f8 [sql generation] fix exists table scope pb: when iterating other solutions inner to an EXISTS subquery, we should reset variables which have this exists node as scope
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6940
diff changeset
   401
        # reset variables which have this exists node as scope at each iteration
8bcf33c493f8 [sql generation] fix exists table scope pb: when iterating other solutions inner to an EXISTS subquery, we should reset variables which have this exists node as scope
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6940
diff changeset
   402
        for var in exists.stmt.defined_vars.itervalues():
8bcf33c493f8 [sql generation] fix exists table scope pb: when iterating other solutions inner to an EXISTS subquery, we should reset variables which have this exists node as scope
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6940
diff changeset
   403
            if var.scope is exists:
8bcf33c493f8 [sql generation] fix exists table scope pb: when iterating other solutions inner to an EXISTS subquery, we should reset variables which have this exists node as scope
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6940
diff changeset
   404
                thisexistsvars.add(var.name)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   405
        origsol = self.solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   406
        origtables = self.tables
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   407
        done = self.done
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   408
        for thisexistssol in thisexistssols:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   409
            for vname in self.unstablevars:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   410
                if thisexistssol[vname] != origsol[vname] and vname in thisexistsvars:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   411
                    break
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   412
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   413
                self.tables = origtables.copy()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   414
                self.solution = thisexistssol
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   415
                yield 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   416
                # cleanup self.done from stuff specific to exists
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
                for var in thisexistsvars:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   418
                    if var in done:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   419
                        done.remove(var)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   420
                for rel in exists.iget_nodes(Relation):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
                    if rel in done:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   422
                        done.remove(rel)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
        self.solution = origsol
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   424
        self.tables = origtables
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   425
5582
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   426
    def push_scope(self, scope_node):
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   427
        self.scope_nodes.append(scope_node)
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   428
        self.scopes[scope_node] = len(self.actual_tables)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   429
        self.actual_tables.append([])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   430
        self._restr_stack.append(self.restrictions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   431
        self.restrictions = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   432
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   433
    def pop_scope(self):
5582
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   434
        del self.scopes[self.scope_nodes[-1]]
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   435
        self.scope_nodes.pop()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   436
        restrictions = self.restrictions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   437
        self.restrictions = self._restr_stack.pop()
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   438
        scope = len(self.actual_tables) - 1
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   439
        # check if we have some outer chain for this scope
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   440
        matching_chains = []
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   441
        for chain in self.outer_chains:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   442
            for tablealias in chain:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   443
                if self.tables[tablealias][0] < scope:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   444
                    # chain belongs to outer scope
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   445
                    break
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   446
            else:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   447
                # chain match current scope
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   448
                matching_chains.append(chain)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   449
        # call to `tables_sql` will pop actual_tables
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   450
        tables = self.tables_sql(matching_chains)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   451
        # cleanup outer join related structure for tables in matching chains
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   452
        for chain in matching_chains:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   453
            self.outer_chains.remove(chain)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   454
            for alias in chain:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   455
                del self.outer_tables[alias]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   456
        return restrictions, tables
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   457
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   458
    # tables handling #########################################################
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   459
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   460
    def add_table(self, table, key=None, scope=-1):
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   461
        if key is None:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   462
            key = table
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   463
        if key in self.tables:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   464
            return
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   465
        if scope < 0:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   466
            scope = len(self.actual_tables) + scope
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   467
        self.tables[key] = (scope, table)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   468
        self.actual_tables[scope].append(table)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   469
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   470
    def alias_and_add_table(self, tablename, scope=-1):
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   471
        alias = '%s%s' % (tablename, self.count)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   472
        self.count += 1
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   473
        self.add_table('%s AS %s' % (tablename, alias), alias, scope)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   474
        return alias
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   475
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   476
    def relation_table(self, relation):
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   477
        """return the table alias used by the given relation"""
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   478
        if relation in self.done:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   479
            return relation._q_sqltable
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   480
        rid = 'rel_%s%s' % (relation.r_type, self.count)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   481
        # relation's table is belonging to the root scope if it is the principal
9324
a5ce068143ad [rql2sql] fix relation table scope when some variable from subquery is used. Closes #3252270
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9230
diff changeset
   482
        # table of one of its variable and that variable belong's to parent
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   483
        # scope
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   484
        for varref in relation.iget_nodes(VariableRef):
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   485
            var = varref.variable
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   486
            # XXX may have a principal without being invariant for this generation,
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   487
            #     not sure this is a pb or not
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   488
            if var.stinfo.get('principal') is relation and var.scope is var.stmt:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   489
                scope = 0
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   490
                break
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   491
        else:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   492
            scope = -1
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   493
        self.count += 1
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   494
        self.add_table('%s_relation AS %s' % (relation.r_type, rid), rid, scope=scope)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   495
        relation._q_sqltable = rid
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   496
        self.done.add(relation)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   497
        return rid
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   498
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   499
    def fti_table(self, relation, fti_table):
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   500
        """return the table alias used by the given has_text relation,
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   501
        `fti_table` being the table name for the plain text index
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   502
        """
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   503
        if relation in self.done:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   504
            try:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   505
                return relation._q_sqltable
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   506
            except AttributeError:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   507
                pass
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   508
        self.done.add(relation)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   509
        scope = self.scopes[relation.scope]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   510
        alias = self.alias_and_add_table(fti_table, scope=scope)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   511
        relation._q_sqltable = alias
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   512
        return alias
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   513
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   514
    # outer join handling ######################################################
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   515
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   516
    def mark_as_used_in_outer_join(self, tablealias, addpending=True):
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   517
        """Mark table of given alias as used in outer join. This must be called
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   518
        after `outer_tables[tablealias]` has been initialized.
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   519
        """
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   520
        # remove a table from actual_table because it's used in an outer join
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   521
        # chain
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   522
        scope, tabledef = self.tables[tablealias]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   523
        self.actual_tables[scope].remove(tabledef)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   524
        # check if there are some pending outer join condition for this table
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   525
        if addpending:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   526
            try:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   527
                pending_conditions = self.outer_pending.pop(tablealias)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   528
            except KeyError:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   529
                pass
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   530
            else:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   531
                self.outer_tables[tablealias][1].extend(pending_conditions)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   532
        else:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   533
            assert not tablealias in self.outer_pending
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   534
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   535
    def add_outer_join_condition(self, tablealias, condition):
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   536
        try:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   537
            outer, conditions, chain = self.outer_tables[tablealias]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   538
            conditions.append(condition)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   539
        except KeyError:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   540
            self.outer_pending.setdefault(tablealias, []).append(condition)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   541
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   542
    def replace_tables_by_outer_join(self, leftalias, rightalias,
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   543
                                     outertype, condition):
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   544
        """tell we need <leftalias> <outertype> JOIN <rightalias> ON <condition>
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   545
        """
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   546
        assert leftalias != rightalias, leftalias
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   547
        outer_tables = self.outer_tables
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   548
        louter, lconditions, lchain = outer_tables.get(leftalias,
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   549
                                                      (None, None, None))
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   550
        router, rconditions, rchain = outer_tables.get(rightalias,
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   551
                                                      (None, None, None))
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   552
        if lchain is None and rchain is None:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   553
            # create a new outer chaine
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   554
            chain = [leftalias, rightalias]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   555
            outer_tables[leftalias] = (None, [], chain)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   556
            outer_tables[rightalias] = (outertype, [condition], chain)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   557
            self.outer_chains.append(chain)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   558
            self.mark_as_used_in_outer_join(leftalias, addpending=False)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   559
            self.mark_as_used_in_outer_join(rightalias)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   560
        elif lchain is None:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   561
            # [A > B > C] + [D > A] -> [D > A > B > C]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   562
            if rightalias == rchain[0]:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   563
                outer_tables[leftalias] = (None, [], rchain)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   564
                conditions = outer_tables[rightalias][1] + [condition]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   565
                outer_tables[rightalias] = (outertype, conditions, rchain)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   566
                rchain.insert(0, leftalias)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   567
            else:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   568
                # [A > B > C] + [D > B] -> [A > B > C < D]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   569
                if outertype == 'LEFT':
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   570
                    outertype = 'RIGHT'
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   571
                outer_tables[leftalias] = (outertype, [condition], rchain)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   572
                rchain.append(leftalias)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   573
            self.mark_as_used_in_outer_join(leftalias)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   574
        elif rchain is None:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   575
            # [A > B > C] + [B > D] -> [A > B > C > D]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   576
            outer_tables[rightalias] = (outertype, [condition], lchain)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   577
            lchain.append(rightalias)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   578
            self.mark_as_used_in_outer_join(rightalias)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   579
        elif lchain is rchain:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   580
            # already in the same chain, simply check compatibility and append
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   581
            # the condition if it's ok
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   582
            lidx = lchain.index(leftalias)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   583
            ridx = lchain.index(rightalias)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   584
            if (outertype == 'FULL' and router != 'FULL') \
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   585
                   or (lidx < ridx and router != 'LEFT') \
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   586
                   or (ridx < lidx and louter != 'RIGHT'):
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   587
                raise BadRQLQuery()
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   588
            # merge conditions
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   589
            if lidx < ridx:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   590
                rconditions.append(condition)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   591
            else:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   592
                lconditions.append(condition)
7357
5ad3154a8810 [rql2sql] fix bug avoiding outer join relation to be used as a variable principal. Closes #1659395
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7252
diff changeset
   593
        elif louter is None:
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   594
            # merge chains
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   595
            self.outer_chains.remove(lchain)
7357
5ad3154a8810 [rql2sql] fix bug avoiding outer join relation to be used as a variable principal. Closes #1659395
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7252
diff changeset
   596
            rchain += lchain
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   597
            self.mark_as_used_in_outer_join(leftalias)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   598
            for alias, (aouter, aconditions, achain) in outer_tables.iteritems():
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   599
                if achain is lchain:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   600
                    outer_tables[alias] = (aouter, aconditions, rchain)
7357
5ad3154a8810 [rql2sql] fix bug avoiding outer join relation to be used as a variable principal. Closes #1659395
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7252
diff changeset
   601
        else:
5ad3154a8810 [rql2sql] fix bug avoiding outer join relation to be used as a variable principal. Closes #1659395
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7252
diff changeset
   602
            raise BadRQLQuery()
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   603
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   604
    # sql generation helpers ###################################################
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   605
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   606
    def tables_sql(self, outer_chains=None):
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   607
        """generate SQL for FROM clause"""
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   608
        # sort for test predictability
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   609
        tables = sorted(self.actual_tables.pop())
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   610
        # process outer joins
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   611
        if outer_chains is None:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   612
            assert not self.actual_tables, self.actual_tables
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   613
            assert not self.outer_pending
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   614
            outer_chains = self.outer_chains
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   615
        for chain in sorted(outer_chains):
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   616
            tablealias = chain[0]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   617
            outertype, conditions, _ = self.outer_tables[tablealias]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   618
            assert _ is chain, (chain, _)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   619
            assert outertype is None, (chain, self.outer_chains)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   620
            assert not conditions, (chain, self.outer_chains)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   621
            assert len(chain) > 1
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   622
            tabledef = self.tables[tablealias][1]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   623
            outerjoin = [tabledef]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   624
            for tablealias in chain[1:]:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   625
                outertype, conditions, _ = self.outer_tables[tablealias]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   626
                assert _ is chain, (chain, self.outer_chains)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   627
                assert outertype in ('LEFT', 'RIGHT', 'FULL'), (
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   628
                    tablealias, outertype, conditions)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   629
                assert isinstance(conditions, (list)), (
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   630
                    tablealias, outertype, conditions)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   631
                tabledef = self.tables[tablealias][1]
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   632
                outerjoin.append('%s OUTER JOIN %s ON (%s)' % (
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   633
                    outertype, tabledef, ' AND '.join(conditions)))
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   634
            tables.append(' '.join(outerjoin))
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   635
        return ', '.join(tables)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   636
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
   637
5782
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   638
def extract_fake_having_terms(having):
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   639
    """RQL's HAVING may be used to contains stuff that should go in the WHERE
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   640
    clause of the SQL query, due to RQL grammar limitation. Split them...
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   641
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   642
    Return a list nodes that can be ANDed with query's WHERE clause. Having
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   643
    subtrees updated in place.
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   644
    """
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   645
    fakehaving = []
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   646
    for subtree in having:
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   647
        ors, tocheck = set(), []
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   648
        for compnode in subtree.get_nodes(Comparison):
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   649
            for fnode in compnode.get_nodes(Function):
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   650
                if fnode.descr().aggregat:
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   651
                    p = compnode.parent
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   652
                    oor = None
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   653
                    while not isinstance(p, Select):
6929
e9464d975908 [rql2sql] fix bug with NOT in "fake" having terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6493
diff changeset
   654
                        if isinstance(p, (Or, Not)):
5782
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   655
                            oor = p
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   656
                        p = p.parent
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   657
                    if oor is not None:
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   658
                        ors.add(oor)
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   659
                    break
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   660
            else:
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   661
                tocheck.append(compnode)
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   662
        # tocheck hold a set of comparison not implying an aggregat function
7672
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   663
        # put them in fakehaving if they don't share an Or node as ancestor
5782
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   664
        # with another comparison containing an aggregat function
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   665
        for compnode in tocheck:
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   666
            parents = set()
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   667
            p = compnode.parent
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   668
            oor = None
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   669
            while not isinstance(p, Select):
5887
3f55f0f10a22 [rql2sql] test and fix crash when OR in HAVING where both expresion are 'fake' having terms (eg not using aggregat function)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5844
diff changeset
   670
                if p in ors or p is None: # p is None for nodes already in fakehaving
5782
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   671
                    break
6929
e9464d975908 [rql2sql] fix bug with NOT in "fake" having terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6493
diff changeset
   672
                if isinstance(p, (Or, Not)):
5782
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   673
                    oor = p
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   674
                p = p.parent
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   675
            else:
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   676
                node = oor or compnode
5887
3f55f0f10a22 [rql2sql] test and fix crash when OR in HAVING where both expresion are 'fake' having terms (eg not using aggregat function)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5844
diff changeset
   677
                fakehaving.append(node)
3f55f0f10a22 [rql2sql] test and fix crash when OR in HAVING where both expresion are 'fake' having terms (eg not using aggregat function)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5844
diff changeset
   678
                node.parent.remove(node)
5782
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   679
    return fakehaving
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
   680
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   681
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   682
class SQLGenerator(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   683
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   684
    generation of SQL from the fully expanded RQL syntax tree
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   685
    SQL is designed to be used with a CubicWeb SQL schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   686
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   687
    Groups and sort are not handled here since they should not be handled at
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   688
    this level (see cubicweb.server.querier)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
   689
9492
c7fc56eecd1a English typography
Dimitri Papadopoulos <dimitri.papadopoulos@cea.fr>
parents: 9361
diff changeset
   690
    we should not have errors here!
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   691
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   692
    WARNING: a CubicWebSQLGenerator instance is not thread safe, but generate is
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   693
    protected by a lock
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   694
    """
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
   695
5010
b2c5aee8ca3f [cleanup] rename dbms_helper to dbhelper for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
   696
    def __init__(self, schema, dbhelper, attrmap=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   697
        self.schema = schema
5010
b2c5aee8ca3f [cleanup] rename dbms_helper to dbhelper for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
   698
        self.dbhelper = dbhelper
b2c5aee8ca3f [cleanup] rename dbms_helper to dbhelper for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
   699
        self.dbencoding = dbhelper.dbencoding
b2c5aee8ca3f [cleanup] rename dbms_helper to dbhelper for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
   700
        self.keyword_map = {'NOW' : self.dbhelper.sql_current_timestamp,
b2c5aee8ca3f [cleanup] rename dbms_helper to dbhelper for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
   701
                            'TODAY': self.dbhelper.sql_current_date,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   702
                            }
5010
b2c5aee8ca3f [cleanup] rename dbms_helper to dbhelper for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
   703
        if not self.dbhelper.union_parentheses_support:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   704
            self.union_sql = self.noparen_union_sql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   705
        self._lock = threading.Lock()
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
   706
        if attrmap is None:
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
   707
            attrmap = {}
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
   708
        self.attr_map = attrmap
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
   709
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   710
    def generate(self, union, args=None, varmap=None):
8238
087bb529035c [spelling] fix dictionnary -> dictionary typo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7902
diff changeset
   711
        """return SQL queries and a variable dictionary from a RQL syntax tree
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   712
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   713
        :partrqls: a list of couple (rqlst, solutions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   714
        :args: optional dictionary with values of substitutions used in the query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   715
        :varmap: optional dictionary mapping variable name to a special table
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   716
          name, in case the query as to fetch data from temporary tables
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   717
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   718
        return an sql string and a dictionary with substitutions values
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   719
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   720
        if args is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   721
            args = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   722
        if varmap is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   723
            varmap =  {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   724
        self._lock.acquire()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   725
        self._args = args
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   726
        self._varmap = varmap
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   727
        self._query_attrs = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   728
        self._state = None
5582
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   729
        # self._not_scope_offset = 0
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   730
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   731
            # union query for each rqlst / solution
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   732
            sql = self.union_sql(union)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   733
            # we are done
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   734
            return sql, self._query_attrs, self._state.needs_source_cb
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   735
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   736
            self._lock.release()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   737
6491
ee9a10b6620e pylint option update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6297
diff changeset
   738
    def union_sql(self, union, needalias=False): # pylint: disable=E0202
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   739
        if len(union.children) == 1:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   740
            return self.select_sql(union.children[0], needalias)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   741
        sqls = ('(%s)' % self.select_sql(select, needalias)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   742
                for select in union.children)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   743
        return '\nUNION ALL\n'.join(sqls)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   744
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   745
    def noparen_union_sql(self, union, needalias=False):
5010
b2c5aee8ca3f [cleanup] rename dbms_helper to dbhelper for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
   746
        # needed for sqlite backend which doesn't like parentheses around union
b2c5aee8ca3f [cleanup] rename dbms_helper to dbhelper for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
   747
        # query. This may cause bug in some condition (sort in one of the
b2c5aee8ca3f [cleanup] rename dbms_helper to dbhelper for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
   748
        # subquery) but will work in most case
b2c5aee8ca3f [cleanup] rename dbms_helper to dbhelper for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
   749
        #
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   750
        # see http://www.sqlite.org/cvstrac/tktview?tn=3074
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   751
        sqls = (self.select_sql(select, needalias)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   752
                for i, select in enumerate(union.children))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   753
        return '\nUNION ALL\n'.join(sqls)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
   754
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   755
    def select_sql(self, select, needalias=False):
8238
087bb529035c [spelling] fix dictionnary -> dictionary typo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7902
diff changeset
   756
        """return SQL queries and a variable dictionary from a RQL syntax tree
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   757
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   758
        :select: a selection statement of the syntax tree (`rql.stmts.Select`)
8238
087bb529035c [spelling] fix dictionnary -> dictionary typo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7902
diff changeset
   759
        :solution: a dictionary containing variables binding.
087bb529035c [spelling] fix dictionnary -> dictionary typo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7902
diff changeset
   760
          A solution's dictionary has variable's names as key and variable's
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   761
          types as values
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   762
        :needwrap: boolean telling if the query will be wrapped in an outer
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   763
          query (to deal with aggregat and/or grouping)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   764
        """
9518
54ead5f372bb [rql2sql] remove special behaviour of symmetric relation vs DISTINCT
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9492
diff changeset
   765
        if select.distinct:
54ead5f372bb [rql2sql] remove special behaviour of symmetric relation vs DISTINCT
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9492
diff changeset
   766
            distinct = True
54ead5f372bb [rql2sql] remove special behaviour of symmetric relation vs DISTINCT
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9492
diff changeset
   767
        elif self.dbhelper.fti_need_distinct:
54ead5f372bb [rql2sql] remove special behaviour of symmetric relation vs DISTINCT
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9492
diff changeset
   768
            distinct = getattr(select.parent, 'has_text_query', False)
54ead5f372bb [rql2sql] remove special behaviour of symmetric relation vs DISTINCT
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9492
diff changeset
   769
        else:
54ead5f372bb [rql2sql] remove special behaviour of symmetric relation vs DISTINCT
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9492
diff changeset
   770
            distinct = False
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   771
        sorts = select.orderby
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   772
        groups = select.groupby
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   773
        having = select.having
7672
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   774
        for restr in extract_fake_having_terms(having):
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   775
            scope = None
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   776
            for vref in restr.get_nodes(VariableRef):
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   777
                vscope = vref.variable.scope
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   778
                if vscope is select:
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   779
                    continue # ignore select scope, so restriction is added to
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   780
                             # the inner most scope possible
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   781
                if scope is None:
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   782
                    scope = vscope
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   783
                elif vscope is not scope:
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   784
                    scope = common_parent(scope, vscope).scope
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   785
            if scope is None:
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   786
                scope = select
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   787
            scope.add_restriction(restr)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   788
        # remember selection, it may be changed and have to be restored
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   789
        origselection = select.selection[:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   790
        # check if the query will have union subquery, if it need sort term
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   791
        # selection (union or distinct query) and wrapping (union with groups)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   792
        needwrap = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   793
        sols = select.solutions
9518
54ead5f372bb [rql2sql] remove special behaviour of symmetric relation vs DISTINCT
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9492
diff changeset
   794
        selectsortterms = distinct
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   795
        if len(sols) > 1:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   796
            # remove invariant from solutions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   797
            sols, existssols, unstable = remove_unused_solutions(
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   798
                select, sols, self._varmap, self.schema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   799
            if len(sols) > 1:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   800
                # if there is still more than one solution, a UNION will be
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   801
                # generated and so sort terms have to be selected
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   802
                selectsortterms = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   803
                # and if select is using group by or aggregat, a wrapping
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   804
                # query will be necessary
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   805
                if groups or select.has_aggregat:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   806
                    select.select_only_variables()
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
   807
                    needwrap = True
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   808
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   809
            existssols, unstable = {}, ()
5582
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   810
        state = StateInfo(select, existssols, unstable)
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   811
        if self._state is not None:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   812
            # state from a previous unioned select
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   813
            state.merge_source_cbs(self._state.needs_source_cb)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   814
        # treat subqueries
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   815
        self._subqueries_sql(select, state)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   816
        # generate sql for this select node
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   817
        if needwrap:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   818
            outerselection = origselection[:]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   819
            if sorts and selectsortterms:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   820
                if distinct:
7579
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   821
                    sort_term_selection(sorts, outerselection, groups)
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   822
        fix_selection_and_group(select, needwrap, selectsortterms,
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   823
                                sorts, groups, having)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   824
        if needwrap:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   825
            fneedwrap = len(outerselection) != len(origselection)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   826
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   827
            fneedwrap = len(select.selection) != len(origselection)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   828
        if fneedwrap:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   829
            needalias = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   830
        self._in_wrapping_query = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   831
        self._state = state
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   832
        try:
7672
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   833
            sql = self._solutions_sql(select, sols, distinct,
5782
8ff48d1a319f [rql2sql] when using HAVING to by-pass rql limitation (not to filter on result of an aggregat function), we should emit SQL that doesn't use HAVING to avoid potential backend error because variables are not grouped. Closes #1061603.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   834
                                      needalias or needwrap)
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   835
            # generate groups / having before wrapping query selection to get
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   836
            # correct column aliases
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   837
            self._in_wrapping_query = needwrap
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   838
            if groups:
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   839
                # no constant should be inserted in GROUP BY else the backend
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   840
                # will interpret it as a positional index in the selection
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   841
                groups = ','.join(vref.accept(self) for vref in groups
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   842
                                  if not isinstance(vref, Constant))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   843
            if having:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   844
                # filter out constants as for GROUP BY
5662
785837baabba HAVING sql clause don't know about comma...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5635
diff changeset
   845
                having = ' AND '.join(term.accept(self) for term in having
785837baabba HAVING sql clause don't know about comma...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5635
diff changeset
   846
                                      if not isinstance(term, Constant))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   847
            if needwrap:
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   848
                sql = '%s FROM (%s) AS T1' % (
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   849
                    self._selection_sql(outerselection, distinct,needalias),
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   850
                    sql)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   851
            if groups:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   852
                sql += '\nGROUP BY %s' % groups
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   853
            if having:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   854
                sql += '\nHAVING %s' % having
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   855
            # sort
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   856
            if sorts:
7108
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7042
diff changeset
   857
                sqlsortterms = []
7579
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   858
                if needwrap:
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   859
                    selectidx = [str(term) for term in outerselection]
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   860
                else:
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   861
                    selectidx = [str(term) for term in select.selection]
7108
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7042
diff changeset
   862
                for sortterm in sorts:
7579
5a610b34d2d2 [rql2sql] more cases fixed where something is wrongly added to GROUPBY, causing unexpected results for the query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7472
diff changeset
   863
                    _term = self._sortterm_sql(sortterm, selectidx)
7108
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7042
diff changeset
   864
                    if _term is not None:
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7042
diff changeset
   865
                        sqlsortterms.append(_term)
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   866
                if sqlsortterms:
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   867
                    sql = self.dbhelper.sql_add_order_by(
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   868
                        sql, sqlsortterms, origselection, fneedwrap,
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   869
                        select.limit or select.offset)
7108
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7042
diff changeset
   870
            else:
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7042
diff changeset
   871
                sqlsortterms = None
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   872
            state.finalize_source_cbs()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   873
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   874
            select.selection = origselection
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   875
        # limit / offset
7108
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7042
diff changeset
   876
        sql = self.dbhelper.sql_add_limit_offset(sql,
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7042
diff changeset
   877
                                                 select.limit,
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7042
diff changeset
   878
                                                 select.offset,
bcdf22734059 Abstract the support for ORDER BY and LIMIT/OFFSET SQL generation
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7042
diff changeset
   879
                                                 sqlsortterms)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   880
        return sql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   881
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   882
    def _subqueries_sql(self, select, state):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   883
        for i, subquery in enumerate(select.with_):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   884
            sql = self.union_sql(subquery.query, needalias=True)
5010
b2c5aee8ca3f [cleanup] rename dbms_helper to dbhelper for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
   885
            tablealias = '_T%s' % i # XXX nested subqueries
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   886
            sql = '(%s) AS %s' % (sql, tablealias)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   887
            state.subtables[tablealias] = (0, sql)
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   888
            latest_state = self._state
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   889
            for vref in subquery.aliases:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   890
                alias = vref.variable
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   891
                alias._q_sqltable = tablealias
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   892
                alias._q_sql = '%s.C%s' % (tablealias, alias.colnum)
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   893
                try:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   894
                    stack = latest_state.needs_source_cb[alias.colnum]
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   895
                    if state.subquery_source_cb is None:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   896
                        state.subquery_source_cb = {}
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   897
                    for selectidx, vref in iter_mapped_var_sels(select, alias):
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   898
                        stack = stack[:]
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   899
                        update_source_cb_stack(state, select, vref, stack)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   900
                        state.subquery_source_cb[selectidx] = stack
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   901
                except KeyError:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   902
                    continue
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   903
7672
f31f9882c90f [rql2sql] closes #1832859: fake HAVING terms w/ EXISTS terms
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7642
diff changeset
   904
    def _solutions_sql(self, select, solutions, distinct, needalias):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   905
        sqls = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   906
        for solution in solutions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   907
            self._state.reset(solution)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   908
            # visit restriction subtree
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   909
            if select.where is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   910
                self._state.add_restriction(select.where.accept(self))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   911
            sql = [self._selection_sql(select.selection, distinct, needalias)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   912
            if self._state.restrictions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   913
                sql.append('WHERE %s' % ' AND '.join(self._state.restrictions))
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
   914
            self._state.merge_source_cbs(self._state._needs_source_cb)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   915
            # add required tables
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   916
            assert len(self._state.actual_tables) == 1, self._state.actual_tables
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   917
            tables = self._state.tables_sql()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   918
            if tables:
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
   919
                sql.insert(1, 'FROM %s' % tables)
5010
b2c5aee8ca3f [cleanup] rename dbms_helper to dbhelper for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
   920
            elif self._state.restrictions and self.dbhelper.needs_from_clause:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   921
                sql.insert(1, 'FROM (SELECT 1) AS _T')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   922
            sqls.append('\n'.join(sql))
5582
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   923
        if distinct:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   924
            return '\nUNION\n'.join(sqls)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   925
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   926
            return '\nUNION ALL\n'.join(sqls)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
   927
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   928
    def _selection_sql(self, selected, distinct, needaliasing=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   929
        clause = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   930
        for term in selected:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   931
            sql = term.accept(self)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   932
            if needaliasing:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   933
                colalias = 'C%s' % len(clause)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   934
                clause.append('%s AS %s' % (sql, colalias))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   935
                if isinstance(term, VariableRef):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   936
                    self._state.aliases[term.name] = colalias
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   937
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   938
                clause.append(sql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   939
        if distinct:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   940
            return 'SELECT DISTINCT %s' % ', '.join(clause)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   941
        return 'SELECT %s' % ', '.join(clause)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   942
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   943
    def _sortterm_sql(self, sortterm, selectidx):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   944
        term = sortterm.term
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   945
        try:
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   946
            sqlterm = selectidx.index(str(term)) + 1
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   947
        except ValueError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   948
            # Constant node or non selected term
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   949
            sqlterm = term.accept(self)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   950
            if sqlterm is None:
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   951
                return None
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   952
        if sortterm.asc:
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
   953
            return str(sqlterm)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   954
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   955
            return '%s DESC' % sqlterm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   956
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   957
    def visit_and(self, et):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   958
        """generate SQL for a AND subtree"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   959
        res = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   960
        for c in et.children:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   961
            part = c.accept(self)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   962
            if part:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   963
                res.append(part)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   964
        return ' AND '.join(res)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   965
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   966
    def visit_or(self, ou):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   967
        """generate SQL for a OR subtree"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   968
        res = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   969
        for c in ou.children:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   970
            part = c.accept(self)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   971
            if part:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   972
                res.append('(%s)' % part)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   973
        if res:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   974
            if len(res) > 1:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   975
                return '(%s)' % ' OR '.join(res)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   976
            return res[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   977
        return ''
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
   978
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   979
    def visit_not(self, node):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   980
        csql = node.children[0].accept(self)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   981
        if node in self._state.done or not csql:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   982
            # already processed or no sql generated by children
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   983
            return csql
5582
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   984
        return 'NOT (%s)' % csql
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   985
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   986
    def visit_exists(self, exists):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   987
        """generate SQL name for a exists subquery"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   988
        sqls = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   989
        for dummy in self._state.iter_exists_sols(exists):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   990
            sql = self._visit_exists(exists)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   991
            if sql:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   992
                sqls.append(sql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   993
        if not sqls:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   994
            return ''
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   995
        return 'EXISTS(%s)' % ' UNION '.join(sqls)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
   996
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   997
    def _visit_exists(self, exists):
5582
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   998
        self._state.push_scope(exists)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   999
        restriction = exists.children[0].accept(self)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1000
        restrictions, tables = self._state.pop_scope()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1001
        if restriction:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1002
            restrictions.append(restriction)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1003
        restriction = ' AND '.join(restrictions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1004
        if not restriction:
6297
23c1e50ff97b [rql] fix bug with query like 'Any 1 WHERE NOT X in_group G': tables should be kept in EXISTS() even when there are no restriction
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6286
diff changeset
  1005
            if tables:
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1006
                return 'SELECT 1 FROM %s' % tables
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1007
            return ''
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1008
        if not tables:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1009
            # XXX could leave surrounding EXISTS() in this case no?
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1010
            sql = 'SELECT 1 WHERE %s' % restriction
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1011
        else:
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1012
            sql = 'SELECT 1 FROM %s WHERE %s' % (tables, restriction)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1013
        return sql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1014
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1015
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1016
    def visit_relation(self, relation):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1017
        """generate SQL for a relation"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1018
        rtype = relation.r_type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1019
        # don't care of type constraint statement (i.e. relation_type = 'is')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1020
        if relation.is_types_restriction():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1021
            return ''
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1022
        lhs, rhs = relation.get_parts()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1023
        rschema = self.schema.rschema(rtype)
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3245
diff changeset
  1024
        if rschema.final:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1025
            if rtype == 'eid' and lhs.variable._q_invariant and \
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1026
                   lhs.variable.stinfo['constnode']:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1027
                # special case where this restriction is already generated by
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1028
                # some other relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1029
                return ''
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1030
            # attribute relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1031
            if rtype == 'has_text':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1032
                sql = self._visit_has_text_relation(relation)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1033
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1034
                rhs_vars = rhs.get_nodes(VariableRef)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1035
                if rhs_vars:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1036
                    # if variable(s) in the RHS
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1037
                    sql = self._visit_var_attr_relation(relation, rhs_vars)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1038
                else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1039
                    # no variables in the RHS
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1040
                    sql = self._visit_attribute_relation(relation)
5706
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1041
        elif (rtype == 'is' and isinstance(rhs.children[0], Constant)
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1042
              and rhs.children[0].eval(self._args) is None):
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1043
            # special case "C is NULL"
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1044
            if lhs.name in self._varmap:
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1045
                lhssql = self._varmap[lhs.name]
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1046
            else:
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1047
                lhssql = lhs.accept(self)
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1048
            return '%s%s' % (lhssql, rhs.accept(self))
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1049
        elif '%s.%s' % (lhs, relation.r_type) in self._varmap:
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1050
            # relation has already been processed by a previous step
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1051
            return ''
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1052
        elif relation.optional:
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1053
            # OPTIONAL relation, generate a left|right outer join
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1054
            if rtype == 'identity' or rschema.inlined:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1055
                sql = self._visit_outer_join_inlined_relation(relation, rschema)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1056
            else:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1057
                sql = self._visit_outer_join_relation(relation, rschema)
5706
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1058
        elif rschema.inlined:
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1059
            sql = self._visit_inlined_relation(relation)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1060
        else:
5706
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1061
            # regular (non final) relation
c2e8290bc7b7 [rql2sql] fix special 'X relation NULL' case (used by security insertion), broken by recent removal of IS operator in RQL
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5702
diff changeset
  1062
            sql = self._visit_relation(relation, rschema)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1063
        return sql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1064
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1065
    def _visit_inlined_relation(self, relation):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1066
        lhsvar, _, rhsvar, rhsconst = relation_info(relation)
6003
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1067
        # we are sure lhsvar is not None
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1068
        lhssql = self._inlined_var_sql(lhsvar, relation.r_type)
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1069
        if rhsvar is None:
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1070
            moresql = None
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1071
        else:
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1072
            moresql = self._extra_join_sql(relation, lhssql, rhsvar)
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1073
        if isinstance(relation.parent, Not):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1074
            self._state.done.add(relation.parent)
6003
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1075
            if rhsvar is not None and rhsvar._q_invariant:
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1076
                sql = '%s IS NULL' % lhssql
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1077
            else:
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1078
                # column != 1234 may not get back rows where column is NULL...
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1079
                sql = '(%s IS NULL OR %s!=%s)' % (
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1080
                    lhssql, lhssql, (rhsvar or rhsconst).accept(self))
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1081
        elif rhsconst is not None:
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1082
            sql = '%s=%s' % (lhssql, rhsconst.accept(self))
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1083
        elif isinstance(rhsvar, Variable) and rhsvar._q_invariant and \
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1084
                 not rhsvar.name in self._varmap:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1085
            # if the rhs variable is only linked to this relation, this mean we
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1086
            # only want the relation to exists, eg NOT NULL in case of inlined
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1087
            # relation
6003
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1088
            if moresql is not None:
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1089
                return moresql
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1090
            return '%s IS NOT NULL' % lhssql
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1091
        else:
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1092
            sql = '%s=%s' % (lhssql, rhsvar.accept(self))
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1093
        if moresql is None:
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1094
            return sql
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1095
        return '%s AND %s' % (sql, moresql)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1096
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1097
    def _process_relation_term(self, relation, rid, termvar, termconst, relfield):
4794
81075b09ebf1 [rql2sql] fix potential crash when testing _q_invariant on a ColumnAlias instead of a Variable by always considering _q_invariant to false for them
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
  1098
        if termconst or not termvar._q_invariant:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1099
            termsql = termconst and termconst.accept(self) or termvar.accept(self)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1100
            yield '%s.%s=%s' % (rid, relfield, termsql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1101
        elif termvar._q_invariant:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1102
            # if the variable is mapped, generate restriction anyway
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1103
            if termvar.name in self._varmap:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1104
                termsql = termvar.accept(self)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1105
                yield '%s.%s=%s' % (rid, relfield, termsql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1106
            extrajoin = self._extra_join_sql(relation, '%s.%s' % (rid, relfield), termvar)
6003
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1107
            if extrajoin is not None:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1108
                yield extrajoin
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1109
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1110
    def _visit_relation(self, relation, rschema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1111
        """generate SQL for a relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1112
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1113
        implements optimization 1.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1114
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1115
        if relation.r_type == 'identity':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1116
            # special case "X identity Y"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1117
            lhs, rhs = relation.get_parts()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1118
            return '%s%s' % (lhs.accept(self), rhs.accept(self))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1119
        lhsvar, lhsconst, rhsvar, rhsconst = relation_info(relation)
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1120
        rid = self._state.relation_table(relation)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1121
        sqls = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1122
        sqls += self._process_relation_term(relation, rid, lhsvar, lhsconst, 'eid_from')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1123
        sqls += self._process_relation_term(relation, rid, rhsvar, rhsconst, 'eid_to')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1124
        sql = ' AND '.join(sqls)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1125
        return sql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1126
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1127
    def _visit_outer_join_relation(self, relation, rschema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1128
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1129
        left outer join syntax (optional=='right'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1130
          X relation Y?
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1131
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1132
        right outer join syntax (optional=='left'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1133
          X? relation Y
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1134
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1135
        full outer join syntaxes (optional=='both'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1136
          X? relation Y?
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1137
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1138
        if relation is inlined:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1139
           if it's a left outer join:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1140
           -> X LEFT OUTER JOIN Y ON (X.relation=Y.eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1141
           elif it's a right outer join:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1142
           -> Y LEFT OUTER JOIN X ON (X.relation=Y.eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1143
           elif it's a full outer join:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1144
           -> X FULL OUTER JOIN Y ON (X.relation=Y.eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1145
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1146
           if it's a left outer join:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1147
           -> X LEFT OUTER JOIN relation ON (relation.eid_from=X.eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1148
              LEFT OUTER JOIN Y ON (relation.eid_to=Y.eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1149
           elif it's a right outer join:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1150
           -> Y LEFT OUTER JOIN relation ON (relation.eid_to=Y.eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1151
              LEFT OUTER JOIN X ON (relation.eid_from=X.eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1152
           elif it's a full outer join:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1153
           -> X FULL OUTER JOIN Y ON (X.relation=Y.eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1154
        """
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1155
        leftvar, leftconst, rightvar, rightconst = relation_info(relation)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1156
        assert not (leftconst and rightconst), "doesn't make sense"
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1157
        if relation.optional == 'left':
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1158
            leftvar, rightvar = rightvar, leftvar
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1159
            leftconst, rightconst = rightconst, leftconst
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1160
            joinattr, restrattr = 'eid_to', 'eid_from'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1161
        else:
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1162
            joinattr, restrattr = 'eid_from', 'eid_to'
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1163
        # search table for this variable, to use as left table of the outer join
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1164
        leftalias = None
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1165
        if leftvar:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1166
            # take care, may return None for invariant variable
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1167
            leftalias = self._var_table(leftvar)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1168
        if leftalias is None:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1169
            if leftvar.stinfo['principal'] is not relation:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1170
                # use variable's principal relation
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1171
                leftalias = leftvar.stinfo['principal']._q_sqltable
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1172
            else:
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1173
                # search for relation on which we should join
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1174
                for orelation in leftvar.stinfo['relations']:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1175
                    if (orelation is not relation and
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1176
                        not self.schema.rschema(orelation.r_type).final):
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1177
                        break
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1178
                else:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1179
                    for orelation in rightvar.stinfo['relations']:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1180
                        if (orelation is not relation and
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1181
                            not self.schema.rschema(orelation.r_type).final
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1182
                            and orelation.optional):
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1183
                            break
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1184
                    else:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1185
                        # unexpected
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1186
                        assert False, leftvar
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1187
                leftalias = self._state.relation_table(orelation)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1188
        # right table of the outer join
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1189
        rightalias = self._state.relation_table(relation)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1190
        # compute join condition
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1191
        if not leftconst or (leftvar and not leftvar._q_invariant):
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1192
            leftsql = leftvar.accept(self)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1193
        else:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1194
            leftsql = leftconst.accept(self)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1195
        condition = '%s.%s=%s' % (rightalias, joinattr, leftsql)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1196
        if rightconst:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1197
            condition += ' AND %s.%s=%s' % (rightalias, restrattr, rightconst.accept(self))
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1198
        # record outer join
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1199
        outertype = 'FULL' if relation.optional == 'both' else 'LEFT'
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1200
        self._state.replace_tables_by_outer_join(leftalias, rightalias,
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1201
                                                 outertype, condition)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1202
        # need another join?
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1203
        if rightconst is None:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1204
            # we need another outer join for the other side of the relation (e.g.
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1205
            # for "X relation Y?" in RQL, we treated earlier the (cw_X.eid /
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1206
            # relation.eid_from) join, now we've to do (relation.eid_to /
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1207
            # cw_Y.eid)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1208
            leftalias = rightalias
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1209
            rightsql = rightvar.accept(self) # accept before using var_table
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1210
            rightalias = self._var_table(rightvar)
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1211
            if rightalias is None:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1212
                if rightvar.stinfo['principal'] is not relation:
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1213
                    self._state.replace_tables_by_outer_join(
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1214
                        leftalias, rightvar.stinfo['principal']._q_sqltable,
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1215
                        outertype, '%s.%s=%s' % (leftalias, restrattr, rightvar.accept(self)))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1216
            else:
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1217
                self._state.replace_tables_by_outer_join(
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1218
                    leftalias, rightalias, outertype,
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1219
                    '%s.%s=%s' % (leftalias, restrattr, rightvar.accept(self)))
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1220
        # this relation will hence be expressed in FROM clause, return nothing
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1221
        # here
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1222
        return ''
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1223
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1224
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1225
    def _visit_outer_join_inlined_relation(self, relation, rschema):
7734
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1226
        lhsvar, lhsconst, rhsvar, rhsconst = relation_info(relation)
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1227
        assert not (lhsconst and rhsconst), "doesn't make sense"
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1228
        attr = 'eid' if relation.r_type == 'identity' else relation.r_type
7734
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1229
        lhsalias = self._var_table(lhsvar)
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1230
        rhsalias = rhsvar and self._var_table(rhsvar)
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1231
        try:
7734
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1232
            lhssql = self._varmap['%s.%s' % (lhsvar.name, attr)]
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1233
        except KeyError:
7734
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1234
            if lhsalias is None:
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1235
                lhssql = lhsconst.accept(self)
9230
a68b2fe8a800 [rql2sql] fix bad sql generated when outer joining 'identity' relation and lhs var comes from a subquery. Closes #3099418
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9167
diff changeset
  1236
            elif attr == 'eid':
a68b2fe8a800 [rql2sql] fix bad sql generated when outer joining 'identity' relation and lhs var comes from a subquery. Closes #3099418
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9167
diff changeset
  1237
                lhssql = lhsvar.accept(self)
7734
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1238
            else:
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1239
                lhssql = '%s.%s%s' % (lhsalias, SQL_PREFIX, attr)
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1240
        condition = '%s=%s' % (lhssql, (rhsconst or rhsvar).accept(self))
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1241
        # this is not a typo, rhs optional variable means lhs outer join and vice-versa
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1242
        if relation.optional == 'left':
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1243
            lhsvar, rhsvar = rhsvar, lhsvar
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1244
            lhsconst, rhsconst = rhsconst, lhsconst
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1245
            lhsalias, rhsalias = rhsalias, lhsalias
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1246
            outertype = 'LEFT'
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1247
        elif relation.optional == 'both':
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1248
            outertype = 'FULL'
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1249
        else:
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1250
            outertype = 'LEFT'
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1251
        if rhsalias is None:
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1252
            if rhsconst is not None:
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1253
                # inlined relation with invariant as rhs
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1254
                if relation.r_type != 'identity':
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1255
                    condition = '(%s OR %s IS NULL)' % (condition, lhssql)
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1256
                if not lhsvar.stinfo.get('optrelations'):
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1257
                    return condition
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1258
                self._state.add_outer_join_condition(lhsalias, condition)
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1259
            return
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1260
        if lhsalias is None:
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1261
            if lhsconst is not None and not rhsvar.stinfo.get('optrelations'):
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1262
                return condition
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1263
            lhsalias = lhsvar._q_sql.split('.', 1)[0]
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1264
        if lhsalias == rhsalias:
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1265
            self._state.add_outer_join_condition(lhsalias, condition)
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1266
        else:
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1267
            self._state.replace_tables_by_outer_join(
59fea81647e5 [rql2sql] closes #1892473: enhance handling of optional inlined relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7729
diff changeset
  1268
                lhsalias, rhsalias, outertype, condition)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1269
        return ''
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1270
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1271
    def _visit_var_attr_relation(self, relation, rhs_vars):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1272
        """visit an attribute relation with variable(s) in the RHS
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1273
8457
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1274
        attribute variables are used either in the selection or for unification
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1275
        (eg X attr1 A, Y attr2 A). In case of selection, nothing to do here.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1276
        """
8457
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1277
        ored = relation.ored()
7472
9833c09460f1 [rql2sql] fix missing sql restriction w/ attribute restriction where value comes from a subquery
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7421
diff changeset
  1278
        for vref in rhs_vars:
9833c09460f1 [rql2sql] fix missing sql restriction w/ attribute restriction where value comes from a subquery
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7421
diff changeset
  1279
            var = vref.variable
2073
173c646981a7 fix missing from close when using a var map
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
  1280
            if var.name in self._varmap:
173c646981a7 fix missing from close when using a var map
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
  1281
                # ensure table is added
7472
9833c09460f1 [rql2sql] fix missing sql restriction w/ attribute restriction where value comes from a subquery
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7421
diff changeset
  1282
                self._var_info(var)
9833c09460f1 [rql2sql] fix missing sql restriction w/ attribute restriction where value comes from a subquery
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7421
diff changeset
  1283
            if isinstance(var, ColumnAlias):
9833c09460f1 [rql2sql] fix missing sql restriction w/ attribute restriction where value comes from a subquery
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7421
diff changeset
  1284
                # force sql generation whatever the computed principal
9833c09460f1 [rql2sql] fix missing sql restriction w/ attribute restriction where value comes from a subquery
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7421
diff changeset
  1285
                principal = 1
9833c09460f1 [rql2sql] fix missing sql restriction w/ attribute restriction where value comes from a subquery
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7421
diff changeset
  1286
            else:
9833c09460f1 [rql2sql] fix missing sql restriction w/ attribute restriction where value comes from a subquery
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7421
diff changeset
  1287
                principal = var.stinfo.get('principal')
8457
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1288
            # we've to return some sql if:
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1289
            # 1. visited relation is ored
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1290
            # 2. variable's principal is not this relation and not 1.
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1291
            if ored or (principal is not None and principal is not relation
8459
1931953226f2 [rql2sql] remove debug print and fix bug introduced in 8469:69ad9a509bc3
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8457
diff changeset
  1292
                        and not getattr(principal, 'ored', lambda : 0)()):
6940
1172c25655b7 [rql2sql] should not select a neged relation as principal + simplify code for sql generation for attribute relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6929
diff changeset
  1293
                # we have to generate unification expression
8457
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1294
                if principal is relation:
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1295
                    # take care if ored case and principal is the relation to
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1296
                    # use the right relation in the unification term
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1297
                    _rel = [rel for rel in var.stinfo['rhsrelations']
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1298
                            if not rel is principal][0]
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1299
                else:
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1300
                    _rel = relation
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1301
                lhssql = self._inlined_var_sql(_rel.children[0].variable,
69ad9a509bc3 [rql2sql] fix generated SQL when an attribute relation with a variable is ored with another attribute relation. Closes #2410405
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8245
diff changeset
  1302
                                               _rel.r_type)
6940
1172c25655b7 [rql2sql] should not select a neged relation as principal + simplify code for sql generation for attribute relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6929
diff changeset
  1303
                try:
1172c25655b7 [rql2sql] should not select a neged relation as principal + simplify code for sql generation for attribute relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6929
diff changeset
  1304
                    self._state.ignore_varmap = True
7707
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1305
                    sql = lhssql + relation.children[1].accept(self)
6940
1172c25655b7 [rql2sql] should not select a neged relation as principal + simplify code for sql generation for attribute relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6929
diff changeset
  1306
                finally:
1172c25655b7 [rql2sql] should not select a neged relation as principal + simplify code for sql generation for attribute relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6929
diff changeset
  1307
                    self._state.ignore_varmap = False
7707
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1308
                if relation.optional == 'right':
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1309
                    leftalias = self._var_table(principal.children[0].variable)
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1310
                    rightalias = self._var_table(relation.children[0].variable)
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1311
                    self._state.replace_tables_by_outer_join(
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1312
                        leftalias, rightalias, 'LEFT', sql)
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1313
                    return ''
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1314
                return sql
6940
1172c25655b7 [rql2sql] should not select a neged relation as principal + simplify code for sql generation for attribute relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6929
diff changeset
  1315
        return ''
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1316
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
  1317
    def _visit_attribute_relation(self, rel):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1318
        """generate SQL for an attribute relation"""
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
  1319
        lhs, rhs = rel.get_parts()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1320
        rhssql = rhs.accept(self)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1321
        table = self._var_table(lhs.variable)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1322
        if table is None:
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
  1323
            assert rel.r_type == 'eid'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1324
            lhssql = lhs.accept(self)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1325
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1326
            try:
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
  1327
                lhssql = self._varmap['%s.%s' % (lhs.name, rel.r_type)]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1328
            except KeyError:
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
  1329
                mapkey = '%s.%s' % (self._state.solution[lhs.name], rel.r_type)
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
  1330
                if mapkey in self.attr_map:
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1331
                    cb, sourcecb = self.attr_map[mapkey]
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1332
                    if sourcecb:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1333
                        # callback is a source callback, we can't use this
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1334
                        # attribute in restriction
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1335
                        raise QueryError("can't use %s (%s) in restriction"
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1336
                                         % (mapkey, rel.as_string()))
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1337
                    lhssql = cb(self, lhs.variable, rel)
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
  1338
                elif rel.r_type == 'eid':
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1124
diff changeset
  1339
                    lhssql = lhs.variable._q_sql
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1124
diff changeset
  1340
                else:
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
  1341
                    lhssql = '%s.%s%s' % (table, SQL_PREFIX, rel.r_type)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1342
        try:
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
  1343
            if rel._q_needcast == 'TODAY':
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1344
                sql = 'DATE(%s)%s' % (lhssql, rhssql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1345
            # XXX which cast function should be used
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
  1346
            #elif rel._q_needcast == 'NOW':
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1347
            #    sql = 'TIMESTAMP(%s)%s' % (lhssql, rhssql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1348
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1349
                sql = '%s%s' % (lhssql, rhssql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1350
        except AttributeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1351
            sql = '%s%s' % (lhssql, rhssql)
5004
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
  1352
        if lhs.variable.stinfo.get('optrelations'):
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1353
            self._state.add_outer_join_condition(table, sql)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1354
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1355
            return sql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1356
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
  1357
    def _visit_has_text_relation(self, rel):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1358
        """generate SQL for a has_text relation"""
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
  1359
        lhs, rhs = rel.get_parts()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1360
        const = rhs.children[0]
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1361
        alias = self._state.fti_table(rel, self.dbhelper.fti_table)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1362
        jointo = lhs.accept(self)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1363
        restriction = ''
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1364
        lhsvar = lhs.variable
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
  1365
        me_is_principal = lhsvar.stinfo.get('principal') is rel
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1366
        if me_is_principal:
5004
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
  1367
            if lhsvar.stinfo['typerel'] is None:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1368
                # the variable is using the fti table, no join needed
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1369
                jointo = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1370
            elif not lhsvar.name in self._varmap:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1371
                # join on entities instead of etype's table to get result for
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1372
                # external entities on multisources configurations
3762
e416186fb91c prefix sql aliases for entity table by '_' to avoid pb with variable such as 'AS' (eg a keyword in SQL)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3754
diff changeset
  1373
                ealias = lhsvar._q_sqltable = '_' + lhsvar.name
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1374
                jointo = lhsvar._q_sql = '%s.eid' % ealias
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1375
                self._state.add_table('entities AS %s' % ealias, ealias)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1376
                if not lhsvar._q_invariant or len(lhsvar.stinfo['possibletypes']) == 1:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1377
                    restriction = " AND %s.type='%s'" % (ealias, self._state.solution[lhs.name])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1378
                else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1379
                    etypes = ','.join("'%s'" % etype for etype in lhsvar.stinfo['possibletypes'])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1380
                    restriction = " AND %s.type IN (%s)" % (ealias, etypes)
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
  1381
        if isinstance(rel.parent, Not):
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2199
diff changeset
  1382
            self._state.done.add(rel.parent)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1383
            not_ = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1384
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1385
            not_ = False
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
  1386
        query = const.eval(self._args)
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
  1387
        return self.dbhelper.fti_restriction_sql(alias, query,
5582
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1388
                                                 jointo, not_) + restriction
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1389
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1390
    def visit_comparison(self, cmp):
3787
82bb2c7f083b C - fix typo. make `lgp check` happy.
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3762
diff changeset
  1391
        """generate SQL for a comparison"""
7707
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1392
        optional = getattr(cmp, 'optional', None) # rql < 0.30
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1393
        if len(cmp.children) == 2:
7707
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1394
            # simplified expression from HAVING clause
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1395
            lhs, rhs = cmp.children
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1396
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1397
            lhs = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1398
            rhs = cmp.children[0]
7707
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1399
            assert not optional
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1400
        sql = None
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1401
        operator = cmp.operator
5702
9fb240cf0f61 [rql] more update to 0.26.2 api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5662
diff changeset
  1402
        if operator in ('LIKE', 'ILIKE'):
5010
b2c5aee8ca3f [cleanup] rename dbms_helper to dbhelper for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
  1403
            if operator == 'ILIKE' and not self.dbhelper.ilike_support:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1404
                operator = ' LIKE '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1405
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1406
                operator = ' %s ' % operator
7257
beea955b45e2 [rql] basic support for regexp-based pattern matching
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7256
diff changeset
  1407
        elif operator == 'REGEXP':
7707
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1408
            sql = ' %s' % self.dbhelper.sql_regexp_match_expression(rhs.accept(self))
1862
94dc8ccd320b #343322: should generate IS NULL in sql w/ None values in substitution
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1802
diff changeset
  1409
        elif (operator == '=' and isinstance(rhs, Constant)
94dc8ccd320b #343322: should generate IS NULL in sql w/ None values in substitution
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1802
diff changeset
  1410
              and rhs.eval(self._args) is None):
94dc8ccd320b #343322: should generate IS NULL in sql w/ None values in substitution
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1802
diff changeset
  1411
            if lhs is None:
7707
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1412
                sql = ' IS NULL'
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1413
            else:
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1414
                sql = '%s IS NULL' % lhs.accept(self)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1415
        elif isinstance(rhs, Function) and rhs.name == 'IN':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1416
            assert operator == '='
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1417
            operator = ' '
7707
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1418
        if sql is None:
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1419
            if lhs is None:
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1420
                sql = '%s%s'% (operator, rhs.accept(self))
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1421
            else:
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1422
                sql = '%s%s%s'% (lhs.accept(self), operator, rhs.accept(self))
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1423
        if optional is None:
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1424
            return sql
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1425
        leftvars = cmp.children[0].get_nodes(VariableRef)
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1426
        assert len(leftvars) == 1
7902
2ada3052e626 [rql2sql] support for outer join on column alias (closes #1979645)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7734
diff changeset
  1427
        if leftvars[0].variable.stinfo['attrvar'] is None:
2ada3052e626 [rql2sql] support for outer join on column alias (closes #1979645)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7734
diff changeset
  1428
            assert isinstance(leftvars[0].variable, ColumnAlias)
2ada3052e626 [rql2sql] support for outer join on column alias (closes #1979645)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7734
diff changeset
  1429
            leftalias = leftvars[0].variable._q_sqltable
2ada3052e626 [rql2sql] support for outer join on column alias (closes #1979645)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7734
diff changeset
  1430
        else:
2ada3052e626 [rql2sql] support for outer join on column alias (closes #1979645)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7734
diff changeset
  1431
            leftalias = self._var_table(leftvars[0].variable.stinfo['attrvar'])
7707
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1432
        rightvars = cmp.children[1].get_nodes(VariableRef)
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1433
        assert len(rightvars) == 1
7902
2ada3052e626 [rql2sql] support for outer join on column alias (closes #1979645)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7734
diff changeset
  1434
        if rightvars[0].variable.stinfo['attrvar'] is None:
2ada3052e626 [rql2sql] support for outer join on column alias (closes #1979645)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7734
diff changeset
  1435
            assert isinstance(rightvars[0].variable, ColumnAlias)
2ada3052e626 [rql2sql] support for outer join on column alias (closes #1979645)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7734
diff changeset
  1436
            rightalias = rightvars[0].variable._q_sqltable
2ada3052e626 [rql2sql] support for outer join on column alias (closes #1979645)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7734
diff changeset
  1437
        else:
2ada3052e626 [rql2sql] support for outer join on column alias (closes #1979645)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7734
diff changeset
  1438
            rightalias = self._var_table(rightvars[0].variable.stinfo['attrvar'])
7707
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1439
        if optional == 'right':
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1440
            self._state.replace_tables_by_outer_join(
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1441
                leftalias, rightalias, 'LEFT', sql)
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1442
        elif optional == 'left':
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1443
            self._state.replace_tables_by_outer_join(
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1444
                rightalias, leftalias, 'LEFT', sql)
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1445
        else:
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1446
            self._state.replace_tables_by_outer_join(
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1447
                leftalias, rightalias, 'FULL', sql)
936530f8d32c [sql gen] handle optional on comparison node (eg HAVING expression) and on rhs of final relation. Closes #1859609
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7704
diff changeset
  1448
        return ''
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1449
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1450
    def visit_mathexpression(self, mexpr):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1451
        """generate SQL for a mathematic expression"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1452
        lhs, rhs = mexpr.get_parts()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1453
        # check for string concatenation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1454
        operator = mexpr.operator
8245
d53762ae5961 [rql2sql] correctly handle modulo operator (closes #2192457)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8238
diff changeset
  1455
        if operator == '%':
d53762ae5961 [rql2sql] correctly handle modulo operator (closes #2192457)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8238
diff changeset
  1456
            operator = '%%'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1457
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1458
            if mexpr.operator == '+' and mexpr.get_type(self._state.solution, self._args) == 'String':
5793
1faff41593df [rql2sql] use new dbhelper.sql_concat_string function to deal with sql string concatenation. This together with lgdb 1.0.4 fixes #1055133: SQL Server support : string concatenation issue
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5782
diff changeset
  1459
                return '(%s)' % self.dbhelper.sql_concat_string(lhs.accept(self),
1faff41593df [rql2sql] use new dbhelper.sql_concat_string function to deal with sql string concatenation. This together with lgdb 1.0.4 fixes #1055133: SQL Server support : string concatenation issue
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5782
diff changeset
  1460
                                                                rhs.accept(self))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1461
        except CoercionError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1462
            pass
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1463
        return '(%s %s %s)'% (lhs.accept(self), operator, rhs.accept(self))
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1464
7729
5f89f11143bc [rql2sql] closes #1892471: support for unary expression introduced in rql 0.30
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7718
diff changeset
  1465
    def visit_unaryexpression(self, uexpr):
5f89f11143bc [rql2sql] closes #1892471: support for unary expression introduced in rql 0.30
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7718
diff changeset
  1466
        """generate SQL for a unary expression"""
5f89f11143bc [rql2sql] closes #1892471: support for unary expression introduced in rql 0.30
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7718
diff changeset
  1467
        return '%s%s'% (uexpr.operator, uexpr.children[0].accept(self))
5f89f11143bc [rql2sql] closes #1892471: support for unary expression introduced in rql 0.30
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7718
diff changeset
  1468
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1469
    def visit_function(self, func):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1470
        """generate SQL name for a function"""
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
  1471
        if func.name == 'FTIRANK':
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
  1472
            try:
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
  1473
                rel = iter(func.children[0].variable.stinfo['ftirels']).next()
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
  1474
            except KeyError:
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
  1475
                raise BadRQLQuery("can't use FTIRANK on variable not used in an"
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
  1476
                                  " 'has_text' relation (eg full-text search)")
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5706
diff changeset
  1477
            const = rel.get_parts()[1].children[0]
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1478
            return self.dbhelper.fti_rank_order(
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1479
                self._state.fti_table(rel, self.dbhelper.fti_table),
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1480
                const.eval(self._args))
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1481
        args = [c.accept(self) for c in func.children]
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1482
        if func in self._state.source_cb_funcs:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1483
            # function executed as a callback on the source
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1484
            assert len(args) == 1
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1485
            return args[0]
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1486
        # func_as_sql will check function is supported by the backend
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1487
        return self.dbhelper.func_as_sql(func.name, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1488
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1489
    def visit_constant(self, constant):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1490
        """generate SQL name for a constant"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1491
        if constant.type is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1492
            return 'NULL'
6286
4d10e9895679 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6131
diff changeset
  1493
        value = constant.value
7256
ccd44caeb936 [rql] support CAST() Function
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7252
diff changeset
  1494
        if constant.type == 'etype':
ccd44caeb936 [rql] support CAST() Function
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7252
diff changeset
  1495
            return value
8894
b03abd90a785 [rql2sql] Remove an XXX by explaining why Int non-substitution is desired
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8893
diff changeset
  1496
        # don't substitute int, causes pb when used as sorting column number
b03abd90a785 [rql2sql] Remove an XXX by explaining why Int non-substitution is desired
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8893
diff changeset
  1497
        if constant.type == 'Int':
7704
0f49dd8e76b5 [rql2sql] don't introduce substitution for Int and Boolean constants. This is needed for COUNT(DISTINCT) support
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7672
diff changeset
  1498
            return str(value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1499
        if constant.type in ('Date', 'Datetime'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1500
            rel = constant.relation()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1501
            if rel is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1502
                rel._q_needcast = value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1503
            return self.keyword_map[value]()
4124
8f2f5f0a89e7 fix backout
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4121
diff changeset
  1504
        if constant.type == 'Substitute':
6131
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1505
            try:
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1506
                # we may found constant from simplified var in varmap
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1507
                return self._mapped_term(constant, '%%(%s)s' % value)[0]
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1508
            except KeyError:
6286
4d10e9895679 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6131
diff changeset
  1509
                _id = value
6131
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1510
                if isinstance(_id, unicode):
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1511
                    _id = _id.encode()
7166
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1512
                # convert timestamp to utc.
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1513
                # expect SET TiME ZONE to UTC at connection opening time.
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1514
                # This shouldn't change anything for datetime without TZ.
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1515
                value = self._args[_id]
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1516
                if isinstance(value, datetime) and value.tzinfo is not None:
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1517
                    self._query_attrs[_id] = utcdatetime(value)
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1518
                elif isinstance(value, time) and value.tzinfo is not None:
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1519
                    self._query_attrs[_id] = utctime(value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1520
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1521
            _id = str(id(constant)).replace('-', '', 1)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1522
            self._query_attrs[_id] = value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1523
        return '%%(%s)s' % _id
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1524
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1525
    def visit_variableref(self, variableref):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1526
        """get the sql name for a variable reference"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1527
        # use accept, .variable may be a variable or a columnalias
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1528
        return variableref.variable.accept(self)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1529
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1530
    def visit_columnalias(self, colalias):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1531
        """get the sql name for a subquery column alias"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1532
        if colalias.name in self._varmap:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1533
            sql = self._varmap[colalias.name]
1122
9f37de24251f fix rql2sq w/ outer join on subquery result
sylvain.thenault@logilab.fr
parents: 438
diff changeset
  1534
            table = sql.split('.', 1)[0]
9f37de24251f fix rql2sq w/ outer join on subquery result
sylvain.thenault@logilab.fr
parents: 438
diff changeset
  1535
            colalias._q_sqltable = table
9f37de24251f fix rql2sq w/ outer join on subquery result
sylvain.thenault@logilab.fr
parents: 438
diff changeset
  1536
            colalias._q_sql = sql
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1537
            self._state.add_table(table)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1538
            return sql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1539
        return colalias._q_sql
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1540
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1541
    def visit_variable(self, variable):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1542
        """get the table name and sql string for a variable"""
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1543
        #if contextrels is None and variable.name in self._state.done:
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1544
        if variable.name in self._state.done:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1545
            if self._in_wrapping_query:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1546
                return 'T1.%s' % self._state.aliases[variable.name]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1547
            return variable._q_sql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1548
        self._state.done.add(variable.name)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1549
        vtablename = None
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1550
        if not self._state.ignore_varmap and variable.name in self._varmap:
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1551
            sql, vtablename = self._var_info(variable)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1552
        elif variable.stinfo['attrvar']:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1553
            # attribute variable (systematically used in rhs of final
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1554
            # relation(s)), get table name and sql from any rhs relation
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1555
            sql = self._linked_var_sql(variable)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1556
        elif variable._q_invariant:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1557
            # since variable is invariant, we know we won't found final relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1558
            principal = variable.stinfo['principal']
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1559
            if principal is None:
3762
e416186fb91c prefix sql aliases for entity table by '_' to avoid pb with variable such as 'AS' (eg a keyword in SQL)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3754
diff changeset
  1560
                vtablename = '_' + variable.name
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1561
                self._state.add_table('entities AS %s' % vtablename, vtablename)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1562
                sql = '%s.eid' % vtablename
5004
4cc020ee70e2 le patch rql26 a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
  1563
                if variable.stinfo['typerel'] is not None:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1564
                    # add additional restriction on entities.type column
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1565
                    pts = variable.stinfo['possibletypes']
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1566
                    if len(pts) == 1:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1567
                        etype = iter(variable.stinfo['possibletypes']).next()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1568
                        restr = "%s.type='%s'" % (vtablename, etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1569
                    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1570
                        etypes = ','.join("'%s'" % et for et in pts)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1571
                        restr = '%s.type IN (%s)' % (vtablename, etypes)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1572
                    self._state.add_restriction(restr)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1573
            elif principal.r_type == 'has_text':
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1574
                sql = '%s.%s' % (self._state.fti_table(principal,
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1575
                                                       self.dbhelper.fti_table),
5010
b2c5aee8ca3f [cleanup] rename dbms_helper to dbhelper for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
  1576
                                 self.dbhelper.fti_uid_attr)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1577
            elif principal in variable.stinfo['rhsrelations']:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1578
                if self.schema.rschema(principal.r_type).inlined:
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1579
                    sql = self._linked_var_sql(variable)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1580
                else:
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1581
                    sql = '%s.eid_to' % self._state.relation_table(principal)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1582
            else:
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1583
                sql = '%s.eid_from' % self._state.relation_table(principal)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1584
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1585
            # standard variable: get table name according to etype and use .eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1586
            # attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1587
            sql, vtablename = self._var_info(variable)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1588
        variable._q_sqltable = vtablename
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1589
        variable._q_sql = sql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1590
        return sql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1591
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1592
    # various utilities #######################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1593
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1594
    def _extra_join_sql(self, relation, sql, var):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1595
        # if rhs var is invariant, and this relation is not its principal,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1596
        # generate extra join
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1597
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1598
            if not var.stinfo['principal'] is relation:
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1599
                op = relation.operator()
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1600
                if op == '=':
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1601
                    # need a predicable result for tests
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1602
                    args = sorted( (sql, var.accept(self)) )
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1603
                    args.insert(1, op)
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1604
                else:
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1605
                    args = (sql, op, var.accept(self))
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1606
                return '%s%s%s' % tuple(args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1607
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1608
            # no principal defined, relation is necessarily the principal and
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1609
            # so nothing to return here
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1610
            pass
6003
5fbc1c4c13ff [rql2sql] fix bug with NOT of inlined relation: NULL values are not properly handled and hence some rows won't be returned while the should
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5901
diff changeset
  1611
        return None
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1612
5593
f6c55bec9326 [rql2sql] properly compute scope for variable from temporary tables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5582
diff changeset
  1613
    def _temp_table_scope(self, select, table):
f6c55bec9326 [rql2sql] properly compute scope for variable from temporary tables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5582
diff changeset
  1614
        scope = 9999
f6c55bec9326 [rql2sql] properly compute scope for variable from temporary tables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5582
diff changeset
  1615
        for var, sql in self._varmap.iteritems():
5635
56784e46509f [rql2sql] fix scope computation of variable from temporary table
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5630
diff changeset
  1616
            # skip "attribute variable" in varmap (such 'T.login')
56784e46509f [rql2sql] fix scope computation of variable from temporary table
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5630
diff changeset
  1617
            if not '.' in var and table == sql.split('.', 1)[0]:
5593
f6c55bec9326 [rql2sql] properly compute scope for variable from temporary tables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5582
diff changeset
  1618
                try:
f6c55bec9326 [rql2sql] properly compute scope for variable from temporary tables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5582
diff changeset
  1619
                    scope = min(scope, self._state.scopes[select.defined_vars[var].scope])
f6c55bec9326 [rql2sql] properly compute scope for variable from temporary tables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5582
diff changeset
  1620
                except KeyError:
f6c55bec9326 [rql2sql] properly compute scope for variable from temporary tables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5582
diff changeset
  1621
                    scope = 0 # XXX
f6c55bec9326 [rql2sql] properly compute scope for variable from temporary tables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5582
diff changeset
  1622
                if scope == 0:
5635
56784e46509f [rql2sql] fix scope computation of variable from temporary table
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5630
diff changeset
  1623
                    break
56784e46509f [rql2sql] fix scope computation of variable from temporary table
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5630
diff changeset
  1624
        return scope
5593
f6c55bec9326 [rql2sql] properly compute scope for variable from temporary tables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5582
diff changeset
  1625
6131
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1626
    def _mapped_term(self, term, key):
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1627
        """return sql and table alias to the `term`, mapped as `key` or raise
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1628
        KeyError when the key is not found in the varmap
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1629
        """
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1630
        sql = self._varmap[key]
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1631
        tablealias = sql.split('.', 1)[0]
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1632
        scope = self._temp_table_scope(term.stmt, tablealias)
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1633
        self._state.add_table(tablealias, scope=scope)
6131
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1634
        return sql, tablealias
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1635
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1636
    def _var_info(self, var):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1637
        try:
6131
087c5a168010 [ms] more planning bug fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6050
diff changeset
  1638
            return self._mapped_term(var, var.name)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1639
        except KeyError:
5593
f6c55bec9326 [rql2sql] properly compute scope for variable from temporary tables
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5582
diff changeset
  1640
            scope = self._state.scopes[var.scope]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1641
            etype = self._state.solution[var.name]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1642
            # XXX this check should be moved in rql.stcheck
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3245
diff changeset
  1643
            if self.schema.eschema(etype).final:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1644
                raise BadRQLQuery(var.stmt.root)
3762
e416186fb91c prefix sql aliases for entity table by '_' to avoid pb with variable such as 'AS' (eg a keyword in SQL)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3754
diff changeset
  1645
            tablealias = '_' + var.name
e416186fb91c prefix sql aliases for entity table by '_' to avoid pb with variable such as 'AS' (eg a keyword in SQL)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3754
diff changeset
  1646
            sql = '%s.%seid' % (tablealias, SQL_PREFIX)
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1647
            self._state.add_table('%s%s AS %s' % (SQL_PREFIX, etype, tablealias),
3762
e416186fb91c prefix sql aliases for entity table by '_' to avoid pb with variable such as 'AS' (eg a keyword in SQL)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3754
diff changeset
  1648
                           tablealias, scope=scope)
e416186fb91c prefix sql aliases for entity table by '_' to avoid pb with variable such as 'AS' (eg a keyword in SQL)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3754
diff changeset
  1649
        return sql, tablealias
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1650
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1651
    def _inlined_var_sql(self, var, rtype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1652
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1653
            sql = self._varmap['%s.%s' % (var.name, rtype)]
5582
3e133b29a1a4 [rql2sql] follow rql 0.26.1 changes: NOT nodes normalization, allowing simplification of sql generation, and fix #XXX
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1654
            scope = self._state.scopes[var.scope]
7193
7eaef037ea9d [sql generation] fix missing join with outer joins w/ rhs invariant variable...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7108
diff changeset
  1655
            self._state.add_table(sql.split('.', 1)[0], scope=scope)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1656
        except KeyError:
7493
3c46b9390871 [rql2sql] fix for some queries where eid variable used and entities picked as variable's table
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7473
diff changeset
  1657
            # rtype may be an attribute relation when called from
3c46b9390871 [rql2sql] fix for some queries where eid variable used and entities picked as variable's table
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7473
diff changeset
  1658
            # _visit_var_attr_relation.  take care about 'eid' rtype, since in
3c46b9390871 [rql2sql] fix for some queries where eid variable used and entities picked as variable's table
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7473
diff changeset
  1659
            # some case we may use the `entities` table, so in that case we've
3c46b9390871 [rql2sql] fix for some queries where eid variable used and entities picked as variable's table
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7473
diff changeset
  1660
            # to properly use variable'sql
3c46b9390871 [rql2sql] fix for some queries where eid variable used and entities picked as variable's table
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7473
diff changeset
  1661
            if rtype == 'eid':
3c46b9390871 [rql2sql] fix for some queries where eid variable used and entities picked as variable's table
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7473
diff changeset
  1662
                sql = var.accept(self)
3c46b9390871 [rql2sql] fix for some queries where eid variable used and entities picked as variable's table
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7473
diff changeset
  1663
            else:
3c46b9390871 [rql2sql] fix for some queries where eid variable used and entities picked as variable's table
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7473
diff changeset
  1664
                sql = '%s.%s%s' % (self._var_table(var), SQL_PREFIX, rtype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1665
        return sql
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1666
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1667
    def _linked_var_sql(self, variable):
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1668
        if not self._state.ignore_varmap:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1669
            try:
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1670
                return self._varmap[variable.name]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1671
            except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1672
                pass
3815
50b87f759b5d test and fix http://www.logilab.org/ticket/499838
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3787
diff changeset
  1673
        rel = (variable.stinfo.get('principal') or
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1674
               iter(variable.stinfo['rhsrelations']).next())
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1675
        linkedvar = rel.children[0].variable
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1676
        if rel.r_type == 'eid':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1677
            return linkedvar.accept(self)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1678
        if isinstance(linkedvar, ColumnAlias):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1679
            raise BadRQLQuery('variable %s should be selected by the subquery'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1680
                              % variable.name)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1681
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1682
            sql = self._varmap['%s.%s' % (linkedvar.name, rel.r_type)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1683
        except KeyError:
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1684
            mapkey = '%s.%s' % (self._state.solution[linkedvar.name], rel.r_type)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1685
            if mapkey in self.attr_map:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1686
                cb, sourcecb = self.attr_map[mapkey]
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1687
                if not sourcecb:
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1688
                    return cb(self, linkedvar, rel)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1689
                # attribute mapped at the source level (bfss for instance)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1690
                stmt = rel.stmt
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1691
                for selectidx, vref in iter_mapped_var_sels(stmt, variable):
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1692
                    stack = [cb]
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1693
                    update_source_cb_stack(self._state, stmt, vref, stack)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5010
diff changeset
  1694
                    self._state._needs_source_cb[selectidx] = stack
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1522
diff changeset
  1695
            linkedvar.accept(self)
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1124
diff changeset
  1696
            sql = '%s.%s%s' % (linkedvar._q_sqltable, SQL_PREFIX, rel.r_type)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1697
        return sql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1698
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1699
    # tables handling #########################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1700
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1701
    def _var_table(self, var):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1702
        var.accept(self)#.visit_variable(var)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1703
        return var._q_sqltable