devtools/repotest.py
author Julien Cristau <julien.cristau@logilab.fr>
Mon, 19 Oct 2015 18:08:34 +0200
changeset 10821 107b87bbbaa7
parent 10804 ee113e1e03de
permissions -rw-r--r--
[devtools] more sorted() vs dict vs python3
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
9833
1485aab7ece6 [tests/querier] use the new connection api (part 3/3)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9831
diff changeset
     1
# copyright 2003-2014 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
"""some utilities to ease repository testing
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
This module contains functions to initialize a new repository.
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
    21
"""
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10365
diff changeset
    22
from __future__ import print_function
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
from pprint import pprint
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
6818
5fa425574548 [server test] can't use skipTest method as a class method, raise appropriate exception instead
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6804
diff changeset
    28
from logilab.common.testlib import SkipTest
2074
9e268cb6202e enhance BasePlannerTC to ease subclasses definition
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2069
diff changeset
    29
9835
5ad968dd9d51 [devtools/repotest] simplify a very small helper
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9834
diff changeset
    30
def tuplify(mylist):
5ad968dd9d51 [devtools/repotest] simplify a very small helper
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9834
diff changeset
    31
    return [tuple(item) for item in mylist]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    32
10711
d7f009a3b960 [py3k] replace cmp with key in sorted()
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10663
diff changeset
    33
def snippet_key(a):
d7f009a3b960 [py3k] replace cmp with key in sorted()
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10663
diff changeset
    34
    # a[0] may be a dict or a key/value tuple
d7f009a3b960 [py3k] replace cmp with key in sorted()
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10663
diff changeset
    35
    return (sorted(dict(a[0]).items()), [e.expression for e in a[1]])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    36
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    37
def test_plan(self, rql, expected, kwargs=None):
9834
807f7a6f33f4 [tests/repotest] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9833
diff changeset
    38
    with self.session.new_cnx() as cnx:
807f7a6f33f4 [tests/repotest] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9833
diff changeset
    39
        plan = self._prepare_plan(cnx, rql, kwargs)
807f7a6f33f4 [tests/repotest] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9833
diff changeset
    40
        self.planner.build_plan(plan)
807f7a6f33f4 [tests/repotest] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9833
diff changeset
    41
        try:
807f7a6f33f4 [tests/repotest] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9833
diff changeset
    42
            self.assertEqual(len(plan.steps), len(expected),
807f7a6f33f4 [tests/repotest] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9833
diff changeset
    43
                              'expected %s steps, got %s' % (len(expected), len(plan.steps)))
807f7a6f33f4 [tests/repotest] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9833
diff changeset
    44
            # step order is important
807f7a6f33f4 [tests/repotest] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9833
diff changeset
    45
            for i, step in enumerate(plan.steps):
807f7a6f33f4 [tests/repotest] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9833
diff changeset
    46
                compare_steps(self, step.test_repr(), expected[i])
807f7a6f33f4 [tests/repotest] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9833
diff changeset
    47
        except AssertionError:
807f7a6f33f4 [tests/repotest] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9833
diff changeset
    48
            pprint([step.test_repr() for step in plan.steps])
807f7a6f33f4 [tests/repotest] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9833
diff changeset
    49
            raise
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    51
def compare_steps(self, step, expected):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
    try:
6369
a151453dc564 [test] more update to unittest2 api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5768
diff changeset
    53
        self.assertEqual(step[0], expected[0], 'expected step type %s, got %s' % (expected[0], step[0]))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    54
        if len(step) > 2 and isinstance(step[1], list) and isinstance(expected[1], list):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    55
            queries, equeries = step[1], expected[1]
6369
a151453dc564 [test] more update to unittest2 api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5768
diff changeset
    56
            self.assertEqual(len(queries), len(equeries),
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    57
                              'expected %s queries, got %s' % (len(equeries), len(queries)))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    58
            for i, (rql, sol) in enumerate(queries):
6369
a151453dc564 [test] more update to unittest2 api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5768
diff changeset
    59
                self.assertEqual(rql, equeries[i][0])
10821
107b87bbbaa7 [devtools] more sorted() vs dict vs python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 10804
diff changeset
    60
                self.assertEqual(sorted(sorted(x.items()) for x in sol), sorted(sorted(x.items()) for x in equeries[i][1]))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    61
            idx = 2
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    62
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
            idx = 1
6369
a151453dc564 [test] more update to unittest2 api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5768
diff changeset
    64
        self.assertEqual(step[idx:-1], expected[idx:-1],
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    65
                          'expected step characteristic \n%s\n, got\n%s' % (expected[1:-1], step[1:-1]))
6369
a151453dc564 [test] more update to unittest2 api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5768
diff changeset
    66
        self.assertEqual(len(step[-1]), len(expected[-1]),
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    67
                          'got %s child steps, expected %s' % (len(step[-1]), len(expected[-1])))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    68
    except AssertionError:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10365
diff changeset
    69
        print('error on step ', end=' ')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    70
        pprint(step[:-1])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    71
        raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    72
    children = step[-1]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    73
    if step[0] in ('UnionFetchStep', 'UnionStep'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    74
        # sort children
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    75
        children = sorted(children)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    76
        expectedchildren = sorted(expected[-1])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    77
    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    78
        expectedchildren = expected[-1]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    79
    for i, substep in enumerate(children):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    80
        compare_steps(self, substep, expectedchildren[i])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    81
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    82
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    83
class DumbOrderedDict(list):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    84
    def __iter__(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    85
        return self.iterkeys()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
    def __contains__(self, key):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
        return key in self.iterkeys()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    88
    def __getitem__(self, key):
1236
aeb46e43138d fix choose_term monkey-patching
sylvain.thenault@logilab.fr
parents: 1235
diff changeset
    89
        for key_, value in list.__iter__(self):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
            if key == key_:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
                return value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
        raise KeyError(key)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
    def iterkeys(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    94
        return (x for x, y in list.__iter__(self))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
    def iteritems(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
        return (x for x in list.__iter__(self))
1236
aeb46e43138d fix choose_term monkey-patching
sylvain.thenault@logilab.fr
parents: 1235
diff changeset
    97
    def items(self):
aeb46e43138d fix choose_term monkey-patching
sylvain.thenault@logilab.fr
parents: 1235
diff changeset
    98
        return [x for x in list.__iter__(self)]
aeb46e43138d fix choose_term monkey-patching
sylvain.thenault@logilab.fr
parents: 1235
diff changeset
    99
aeb46e43138d fix choose_term monkey-patching
sylvain.thenault@logilab.fr
parents: 1235
diff changeset
   100
class DumbOrderedDict2(object):
aeb46e43138d fix choose_term monkey-patching
sylvain.thenault@logilab.fr
parents: 1235
diff changeset
   101
    def __init__(self, origdict, sortkey):
aeb46e43138d fix choose_term monkey-patching
sylvain.thenault@logilab.fr
parents: 1235
diff changeset
   102
        self.origdict = origdict
aeb46e43138d fix choose_term monkey-patching
sylvain.thenault@logilab.fr
parents: 1235
diff changeset
   103
        self.sortkey = sortkey
aeb46e43138d fix choose_term monkey-patching
sylvain.thenault@logilab.fr
parents: 1235
diff changeset
   104
    def __getattr__(self, attr):
aeb46e43138d fix choose_term monkey-patching
sylvain.thenault@logilab.fr
parents: 1235
diff changeset
   105
        return getattr(self.origdict, attr)
aeb46e43138d fix choose_term monkey-patching
sylvain.thenault@logilab.fr
parents: 1235
diff changeset
   106
    def __iter__(self):
aeb46e43138d fix choose_term monkey-patching
sylvain.thenault@logilab.fr
parents: 1235
diff changeset
   107
        return iter(sorted(self.origdict, key=self.sortkey))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   108
4766
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   109
def schema_eids_idx(schema):
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   110
    """return a dictionary mapping schema types to their eids so we can reread
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   111
    it from the fs instead of the db (too costly) between tests
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   112
    """
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   113
    schema_eids = {}
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   114
    for x in schema.entities():
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   115
        schema_eids[x] = x.eid
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   116
    for x in schema.relations():
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   117
        schema_eids[x] = x.eid
10663
54b8a1f249fb [py3k] dict.itervalues → dict.values
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10662
diff changeset
   118
        for rdef in x.rdefs.values():
4766
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   119
            schema_eids[(rdef.subject, rdef.rtype, rdef.object)] = rdef.eid
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   120
    return schema_eids
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   121
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   122
def restore_schema_eids_idx(schema, schema_eids):
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   123
    """rebuild schema eid index"""
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   124
    for x in schema.entities():
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   125
        x.eid = schema_eids[x]
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   126
        schema._eid_index[x.eid] = x
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   127
    for x in schema.relations():
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   128
        x.eid = schema_eids[x]
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   129
        schema._eid_index[x.eid] = x
10663
54b8a1f249fb [py3k] dict.itervalues → dict.values
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10662
diff changeset
   130
        for rdef in x.rdefs.values():
4766
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   131
            rdef.eid = schema_eids[(rdef.subject, rdef.rtype, rdef.object)]
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   132
            schema._eid_index[rdef.eid] = rdef
162b2b127b15 [test] get a chance to get proper garbage collection when running pytest on whole cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4674
diff changeset
   133
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   134
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   135
from logilab.common.testlib import TestCase, mock_object
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   136
from logilab.database import get_db_helper
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   137
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   138
from rql import RQLHelper
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   139
9460
a2a0bc984863 [config] cleanup/refactor server sources file values handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9456
diff changeset
   140
from cubicweb.devtools.fake import FakeRepo, FakeConfig, FakeSession
6758
28b11ecf319b [ms] #1382452: incorrect results with multi-sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6671
diff changeset
   141
from cubicweb.server import set_debug, debugged
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
from cubicweb.server.querier import QuerierHelper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   143
from cubicweb.server.session import Session
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   144
from cubicweb.server.sources.rql2sql import SQLGenerator, remove_unused_solutions
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   145
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   146
class RQLGeneratorTC(TestCase):
7879
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7850
diff changeset
   147
    schema = backend = None # set this in concrete class
6804
4262d0d6abcb unittest2 fix: can't skip test in setUp, but it's fine in setUpClass
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6794
diff changeset
   148
4262d0d6abcb unittest2 fix: can't skip test in setUp, but it's fine in setUpClass
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6794
diff changeset
   149
    @classmethod
4262d0d6abcb unittest2 fix: can't skip test in setUp, but it's fine in setUpClass
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6794
diff changeset
   150
    def setUpClass(cls):
4262d0d6abcb unittest2 fix: can't skip test in setUp, but it's fine in setUpClass
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6794
diff changeset
   151
        if cls.backend is not None:
4262d0d6abcb unittest2 fix: can't skip test in setUp, but it's fine in setUpClass
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6794
diff changeset
   152
            try:
4262d0d6abcb unittest2 fix: can't skip test in setUp, but it's fine in setUpClass
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6794
diff changeset
   153
                cls.dbhelper = get_db_helper(cls.backend)
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8544
diff changeset
   154
            except ImportError as ex:
6818
5fa425574548 [server test] can't use skipTest method as a class method, raise appropriate exception instead
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6804
diff changeset
   155
                raise SkipTest(str(ex))
6804
4262d0d6abcb unittest2 fix: can't skip test in setUp, but it's fine in setUpClass
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6794
diff changeset
   156
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   157
    def setUp(self):
9460
a2a0bc984863 [config] cleanup/refactor server sources file values handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9456
diff changeset
   158
        self.repo = FakeRepo(self.schema, config=FakeConfig(apphome=self.datadir))
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   159
        self.repo.system_source = mock_object(dbdriver=self.backend)
9850
5ef9dd383ae2 [tests/rqlannotation,querier] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9835
diff changeset
   160
        self.rqlhelper = RQLHelper(self.schema,
5ef9dd383ae2 [tests/rqlannotation,querier] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9835
diff changeset
   161
                                   special_relations={'eid': 'uid',
5ef9dd383ae2 [tests/rqlannotation,querier] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9835
diff changeset
   162
                                                      'has_text': 'fti'},
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   163
                                   backend=self.backend)
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2200
diff changeset
   164
        self.qhelper = QuerierHelper(self.repo, self.schema)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   165
        ExecutionPlan._check_permissions = _dummy_check_permissions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   166
        rqlannotation._select_principal = _select_principal
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   167
        if self.backend is not None:
6804
4262d0d6abcb unittest2 fix: can't skip test in setUp, but it's fine in setUpClass
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6794
diff changeset
   168
            self.o = SQLGenerator(self.schema, self.dbhelper)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   169
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   170
    def tearDown(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   171
        ExecutionPlan._check_permissions = _orig_check_permissions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   172
        rqlannotation._select_principal = _orig_select_principal
2172
cf8f9180e63e delete-trailing-whitespace
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2074
diff changeset
   173
2069
7c0a1b90b1c6 set_debug method on rql2sql test as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   174
    def set_debug(self, debug):
7c0a1b90b1c6 set_debug method on rql2sql test as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   175
        set_debug(debug)
6758
28b11ecf319b [ms] #1382452: incorrect results with multi-sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6671
diff changeset
   176
    def debugged(self, debug):
28b11ecf319b [ms] #1382452: incorrect results with multi-sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6671
diff changeset
   177
        return debugged(debug)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1789
diff changeset
   178
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   179
    def _prepare(self, rql):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   180
        #print '******************** prepare', rql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   181
        union = self.rqlhelper.parse(rql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   182
        #print '********* parsed', union.as_string()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   183
        self.rqlhelper.compute_solutions(union)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   184
        #print '********* solutions', solutions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   185
        self.rqlhelper.simplify(union)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   186
        #print '********* simplified', union.as_string()
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2200
diff changeset
   187
        plan = self.qhelper.plan_factory(union, {}, FakeSession(self.repo))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   188
        plan.preprocess(union)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   189
        for select in union.children:
10804
ee113e1e03de [devtools] pass a key to sort() method
Julien Cristau <julien.cristau@logilab.fr>
parents: 10747
diff changeset
   190
            select.solutions.sort(key=lambda x: list(x.items()))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   191
        #print '********* ppsolutions', solutions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   192
        return union
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   193
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   194
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   195
class BaseQuerierTC(TestCase):
7879
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7850
diff changeset
   196
    repo = None # set this in concrete class
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1789
diff changeset
   197
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   198
    def setUp(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   199
        self.o = self.repo.querier
10747
cf711d1c951f [devtools] 'dict_values' object does not support indexing
Julien Cristau <julien.cristau@logilab.fr>
parents: 10711
diff changeset
   200
        self.session = next(iter(self.repo._sessions.values()))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   201
        self.ueid = self.session.user.eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   202
        assert self.ueid != -1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   203
        self.repo._type_source_cache = {} # clear cache
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   204
        self.maxeid = self.get_max_eid()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   205
        do_monkey_patch()
4773
6ab9ca63531f [testlib] properly close dumb sessions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4766
diff changeset
   206
        self._dumb_sessions = []
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   207
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   208
    def get_max_eid(self):
9833
1485aab7ece6 [tests/querier] use the new connection api (part 3/3)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9831
diff changeset
   209
        with self.session.new_cnx() as cnx:
1485aab7ece6 [tests/querier] use the new connection api (part 3/3)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9831
diff changeset
   210
            return cnx.execute('Any MAX(X)')[0][0]
1485aab7ece6 [tests/querier] use the new connection api (part 3/3)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9831
diff changeset
   211
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   212
    def cleanup(self):
9833
1485aab7ece6 [tests/querier] use the new connection api (part 3/3)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9831
diff changeset
   213
        with self.session.new_cnx() as cnx:
1485aab7ece6 [tests/querier] use the new connection api (part 3/3)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9831
diff changeset
   214
            cnx.execute('DELETE Any X WHERE X eid > %s' % self.maxeid)
1485aab7ece6 [tests/querier] use the new connection api (part 3/3)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9831
diff changeset
   215
            cnx.commit()
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1789
diff changeset
   216
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   217
    def tearDown(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   218
        undo_monkey_patch()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   219
        self.cleanup()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   220
        assert self.session.user.eid != -1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   221
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   222
    def set_debug(self, debug):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   223
        set_debug(debug)
6758
28b11ecf319b [ms] #1382452: incorrect results with multi-sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6671
diff changeset
   224
    def debugged(self, debug):
28b11ecf319b [ms] #1382452: incorrect results with multi-sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6671
diff changeset
   225
        return debugged(debug)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1789
diff changeset
   226
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   227
    def _rqlhelper(self):
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2200
diff changeset
   228
        rqlhelper = self.repo.vreg.rqlhelper
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   229
        # reset uid_func so it don't try to get type from eids
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   230
        rqlhelper._analyser.uid_func = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   231
        rqlhelper._analyser.uid_func_mapping = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   232
        return rqlhelper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   233
9850
5ef9dd383ae2 [tests/rqlannotation,querier] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9835
diff changeset
   234
    def _prepare_plan(self, cnx, rql, kwargs=None, simplify=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   235
        rqlhelper = self._rqlhelper()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   236
        rqlst = rqlhelper.parse(rql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   237
        rqlhelper.compute_solutions(rqlst, kwargs=kwargs)
4184
7002e91d304a should not simplify previous to preprocess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3240
diff changeset
   238
        if simplify:
7002e91d304a should not simplify previous to preprocess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3240
diff changeset
   239
            rqlhelper.simplify(rqlst)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   240
        for select in rqlst.children:
10804
ee113e1e03de [devtools] pass a key to sort() method
Julien Cristau <julien.cristau@logilab.fr>
parents: 10747
diff changeset
   241
            select.solutions.sort(key=lambda x: list(x.items()))
9850
5ef9dd383ae2 [tests/rqlannotation,querier] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9835
diff changeset
   242
        return self.o.plan_factory(rqlst, kwargs, cnx)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1789
diff changeset
   243
9850
5ef9dd383ae2 [tests/rqlannotation,querier] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9835
diff changeset
   244
    def _prepare(self, cnx, rql, kwargs=None):
5ef9dd383ae2 [tests/rqlannotation,querier] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9835
diff changeset
   245
        plan = self._prepare_plan(cnx, rql, kwargs, simplify=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   246
        plan.preprocess(plan.rqlst)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   247
        rqlst = plan.rqlst.children[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   248
        rqlst.solutions = remove_unused_solutions(rqlst, rqlst.solutions, {}, self.repo.schema)[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   249
        return rqlst
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   250
4674
3d509dbb473a [test api] rename _user_session to user_groups_session, fix its arguments and return only the session, not (user, session) to make things clearer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   251
    def user_groups_session(self, *groups):
3d509dbb473a [test api] rename _user_session to user_groups_session, fix its arguments and return only the session, not (user, session) to make things clearer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   252
        """lightweight session using the current user with hi-jacked groups"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   253
        # use self.session.user.eid to get correct owned_by relation, unless explicit eid
9833
1485aab7ece6 [tests/querier] use the new connection api (part 3/3)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9831
diff changeset
   254
        with self.session.new_cnx() as cnx:
1485aab7ece6 [tests/querier] use the new connection api (part 3/3)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9831
diff changeset
   255
            u = self.repo._build_user(cnx, self.session.user.eid)
1485aab7ece6 [tests/querier] use the new connection api (part 3/3)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9831
diff changeset
   256
            u._groups = set(groups)
1485aab7ece6 [tests/querier] use the new connection api (part 3/3)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9831
diff changeset
   257
            s = Session(u, self.repo)
1485aab7ece6 [tests/querier] use the new connection api (part 3/3)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9831
diff changeset
   258
            return s
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   259
9831
db4aeba5d336 [test/querier] use the new connection api (part 1/3)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9468
diff changeset
   260
    def qexecute(self, rql, args=None, build_descr=True):
db4aeba5d336 [test/querier] use the new connection api (part 1/3)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9468
diff changeset
   261
        with self.session.new_cnx() as cnx:
10365
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9850
diff changeset
   262
            try:
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9850
diff changeset
   263
                return self.o.execute(cnx, rql, args, build_descr)
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9850
diff changeset
   264
            finally:
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9850
diff changeset
   265
                if rql.startswith(('INSERT', 'DELETE', 'SET')):
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9850
diff changeset
   266
                    cnx.commit()
9831
db4aeba5d336 [test/querier] use the new connection api (part 1/3)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9468
diff changeset
   267
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   268
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   269
class BasePlannerTC(BaseQuerierTC):
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6818
diff changeset
   270
2074
9e268cb6202e enhance BasePlannerTC to ease subclasses definition
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2069
diff changeset
   271
    def setup(self):
9e268cb6202e enhance BasePlannerTC to ease subclasses definition
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2069
diff changeset
   272
        # XXX source_defs
9e268cb6202e enhance BasePlannerTC to ease subclasses definition
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2069
diff changeset
   273
        self.o = self.repo.querier
9e268cb6202e enhance BasePlannerTC to ease subclasses definition
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2069
diff changeset
   274
        self.session = self.repo._sessions.values()[0]
9e268cb6202e enhance BasePlannerTC to ease subclasses definition
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2069
diff changeset
   275
        self.schema = self.o.schema
9456
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9452
diff changeset
   276
        self.system = self.repo.system_source
2074
9e268cb6202e enhance BasePlannerTC to ease subclasses definition
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2069
diff changeset
   277
        do_monkey_patch()
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   278
        self.repo.vreg.rqlhelper.backend = 'postgres' # so FTIRANK is considered
2074
9e268cb6202e enhance BasePlannerTC to ease subclasses definition
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2069
diff changeset
   279
9e268cb6202e enhance BasePlannerTC to ease subclasses definition
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2069
diff changeset
   280
    def tearDown(self):
9e268cb6202e enhance BasePlannerTC to ease subclasses definition
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2069
diff changeset
   281
        undo_monkey_patch()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   282
9834
807f7a6f33f4 [tests/repotest] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9833
diff changeset
   283
    def _prepare_plan(self, cnx, rql, kwargs=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   284
        rqlst = self.o.parse(rql, annotate=True)
9834
807f7a6f33f4 [tests/repotest] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9833
diff changeset
   285
        self.o.solutions(cnx, rqlst, kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   286
        if rqlst.TYPE == 'select':
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2200
diff changeset
   287
            self.repo.vreg.rqlhelper.annotate(rqlst)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   288
            for select in rqlst.children:
10804
ee113e1e03de [devtools] pass a key to sort() method
Julien Cristau <julien.cristau@logilab.fr>
parents: 10747
diff changeset
   289
                select.solutions.sort(key=lambda x: list(x.items()))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
        else:
10804
ee113e1e03de [devtools] pass a key to sort() method
Julien Cristau <julien.cristau@logilab.fr>
parents: 10747
diff changeset
   291
            rqlst.solutions.sort(key=lambda x: list(x.items()))
9834
807f7a6f33f4 [tests/repotest] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9833
diff changeset
   292
        return self.o.plan_factory(rqlst, kwargs, cnx)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   293
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   294
10711
d7f009a3b960 [py3k] replace cmp with key in sorted()
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10663
diff changeset
   295
# monkey patch some methods to get predictable results #######################
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   296
7850
d14b77c42b06 [test] sort to avoid random failure due to dict order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   297
from cubicweb import rqlrewrite
d14b77c42b06 [test] sort to avoid random failure due to dict order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   298
_orig_iter_relations = rqlrewrite.iter_relations
d14b77c42b06 [test] sort to avoid random failure due to dict order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   299
_orig_insert_snippets = rqlrewrite.RQLRewriter.insert_snippets
d14b77c42b06 [test] sort to avoid random failure due to dict order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   300
_orig_build_variantes = rqlrewrite.RQLRewriter.build_variantes
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   301
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   302
def _insert_snippets(self, snippets, varexistsmap=None):
10711
d7f009a3b960 [py3k] replace cmp with key in sorted()
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10663
diff changeset
   303
    _orig_insert_snippets(self, sorted(snippets, key=snippet_key), varexistsmap)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   304
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   305
def _build_variantes(self, newsolutions):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   306
    variantes = _orig_build_variantes(self, newsolutions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   307
    sortedvariantes = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   308
    for variante in variantes:
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   309
        orderedkeys = sorted((k[1], k[2], v) for k, v in variante.items())
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   310
        variante = DumbOrderedDict(sorted(variante.items(),
10711
d7f009a3b960 [py3k] replace cmp with key in sorted()
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10663
diff changeset
   311
                                          key=lambda a: (a[0][1], a[0][2], a[1])))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   312
        sortedvariantes.append( (orderedkeys, variante) )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   313
    return [v for ok, v in sorted(sortedvariantes)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   315
from cubicweb.server.querier import ExecutionPlan
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   316
_orig_check_permissions = ExecutionPlan._check_permissions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   317
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   318
def _check_permissions(*args, **kwargs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   319
    res, restricted = _orig_check_permissions(*args, **kwargs)
10821
107b87bbbaa7 [devtools] more sorted() vs dict vs python3
Julien Cristau <julien.cristau@logilab.fr>
parents: 10804
diff changeset
   320
    res = DumbOrderedDict(sorted(res.items(), key=lambda x: [y.items() for y in x[1]]))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
    return res, restricted
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   322
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
def _dummy_check_permissions(self, rqlst):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
    return {(): rqlst.solutions}, set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
from cubicweb.server import rqlannotation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   327
_orig_select_principal = rqlannotation._select_principal
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
def _select_principal(scope, relations):
7357
5ad3154a8810 [rql2sql] fix bug avoiding outer join relation to be used as a variable principal. Closes #1659395
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
   330
    def sort_key(something):
5ad3154a8810 [rql2sql] fix bug avoiding outer join relation to be used as a variable principal. Closes #1659395
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
   331
        try:
5ad3154a8810 [rql2sql] fix bug avoiding outer join relation to be used as a variable principal. Closes #1659395
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
   332
            return something.r_type
5ad3154a8810 [rql2sql] fix bug avoiding outer join relation to be used as a variable principal. Closes #1659395
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
   333
        except AttributeError:
5ad3154a8810 [rql2sql] fix bug avoiding outer join relation to be used as a variable principal. Closes #1659395
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
   334
            return (something[0].r_type, something[1])
599
9ef680acd92a fix select principal so results are predictable during tests
Sylvain <syt@logilab.fr>
parents: 0
diff changeset
   335
    return _orig_select_principal(scope, relations,
7357
5ad3154a8810 [rql2sql] fix bug avoiding outer join relation to be used as a variable principal. Closes #1659395
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
   336
                                  _sort=lambda rels: sorted(rels, key=sort_key))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   337
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   338
7850
d14b77c42b06 [test] sort to avoid random failure due to dict order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   339
def _ordered_iter_relations(stinfo):
d14b77c42b06 [test] sort to avoid random failure due to dict order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   340
    return sorted(_orig_iter_relations(stinfo), key=lambda x:x.r_type)
d14b77c42b06 [test] sort to avoid random failure due to dict order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   341
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   342
def do_monkey_patch():
7850
d14b77c42b06 [test] sort to avoid random failure due to dict order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   343
    rqlrewrite.iter_relations = _ordered_iter_relations
d14b77c42b06 [test] sort to avoid random failure due to dict order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   344
    rqlrewrite.RQLRewriter.insert_snippets = _insert_snippets
d14b77c42b06 [test] sort to avoid random failure due to dict order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   345
    rqlrewrite.RQLRewriter.build_variantes = _build_variantes
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   346
    ExecutionPlan._check_permissions = _check_permissions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   347
    ExecutionPlan.tablesinorder = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   348
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
def undo_monkey_patch():
7850
d14b77c42b06 [test] sort to avoid random failure due to dict order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   350
    rqlrewrite.iter_relations = _orig_iter_relations
d14b77c42b06 [test] sort to avoid random failure due to dict order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   351
    rqlrewrite.RQLRewriter.insert_snippets = _orig_insert_snippets
d14b77c42b06 [test] sort to avoid random failure due to dict order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   352
    rqlrewrite.RQLRewriter.build_variantes = _orig_build_variantes
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   353
    ExecutionPlan._check_permissions = _orig_check_permissions