cubicweb/devtools/fill.py
author Denis Laxalde <denis.laxalde@logilab.fr>
Sat, 16 Jan 2016 13:48:51 +0100
changeset 11057 0b59724cb3f2
parent 10714 devtools/fill.py@8a2ec43fcf44
child 11129 97095348b3ee
permissions -rw-r--r--
Reorganize source tree to have a "cubicweb" top-level package Basically: mkdir cubicweb hg mv *.py -X setup.py cubicweb hg mv dataimport devtools entities etwist ext hooks i18n misc schemas server skeleton sobjects test web wsgi cubicweb Other changes: * adjust path to cubicweb-ctl in devtools tests * update setup.py to avoid importing __pkginfo__ (exec it instead), replace os.path.walk by os.walk and prepend `modname` here and there * update tox.ini to account for new test locations * update doc/conf.py so that it still finds __pkginfo__.py and CWDIR in doc/Makefile
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     1
# -*- coding: iso-8859-1 -*-
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
     2
# copyright 2003-2014 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
     3
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
     4
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
     5
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
     6
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
     7
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
     8
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
     9
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    10
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    11
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    12
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    13
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    14
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    15
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    16
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    17
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    18
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
5786
7c16a4e7a592 [test] fix stupd name error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    19
"""This modules defines func / methods for creating test repositories"""
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9837
diff changeset
    20
from __future__ import print_function
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
7995
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
    24
import logging
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
from random import randint, choice
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
from copy import deepcopy
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    27
from datetime import datetime, date, time, timedelta
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 428
diff changeset
    28
from decimal import Decimal
10714
8a2ec43fcf44 [py3k] use inspect module to get a function's arguments
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10704
diff changeset
    29
import inspect
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    30
10702
f94c812c3669 [py3k] unicode → six.text_type
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10688
diff changeset
    31
from six import text_type, add_metaclass
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    32
from six.moves import range
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    33
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    34
from logilab.common import attrdict
7995
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
    35
from logilab.mtconverter import xml_escape
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    36
from yams.constraints import (SizeConstraint, StaticVocabularyConstraint,
6916
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
    37
                              IntervalBoundConstraint, BoundaryConstraint,
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    38
                              Attribute, actual_value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    39
from rql.utils import decompose_b26 as base_decompose_b26
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    40
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    41
from cubicweb import Binary
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    42
from cubicweb.schema import RQLConstraint
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    43
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    44
def custom_range(start, stop, step):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    45
    while start < stop:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    46
        yield start
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    47
        start += step
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    48
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    49
def decompose_b26(index, ascii=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
    """return a letter (base-26) decomposition of index"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    51
    if ascii:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
        return base_decompose_b26(index)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    53
    return base_decompose_b26(index, u'�abcdefghijklmnopqrstuvwxyz')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    54
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    55
def get_max_length(eschema, attrname):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    56
    """returns the maximum length allowed for 'attrname'"""
4053
7cc66b1d9183 more api update
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 3749
diff changeset
    57
    for cst in eschema.rdef(attrname).constraints:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    58
        if isinstance(cst, SizeConstraint) and cst.max:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    59
            return cst.max
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    60
    return 300
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    61
    #raise AttributeError('No Size constraint on attribute "%s"' % attrname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    62
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
_GENERATED_VALUES = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    64
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    65
class _ValueGenerator(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    66
    """generates integers / dates / strings / etc. to fill a DB table"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    67
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    68
    def __init__(self, eschema, choice_func=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    69
        """<choice_func> is a function that returns a list of possible
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    70
        choices for a given entity type and an attribute name. It should
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    71
        looks like :
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    72
            def values_for(etype, attrname):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    73
                # some stuff ...
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    74
                return alist_of_acceptable_values # or None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    75
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    76
        self.choice_func = choice_func
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    77
        self.eschema = eschema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    78
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    79
    def generate_attribute_value(self, entity, attrname, index=1, **kwargs):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    80
        if attrname in entity:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    81
            return entity[attrname]
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    82
        eschema = self.eschema
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    83
        if not eschema.has_unique_values(attrname):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    84
            value = self.__generate_value(entity, attrname, index, **kwargs)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    85
        else:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    86
            value = self.__generate_value(entity, attrname, index, **kwargs)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    87
            while value in _GENERATED_VALUES.get((eschema, attrname), ()):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    88
                index += 1
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    89
                value = self.__generate_value(entity, attrname, index, **kwargs)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    90
            _GENERATED_VALUES.setdefault((eschema, attrname), set()).add(value)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    91
        entity[attrname] = value
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
        return value
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
    93
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    94
    def __generate_value(self, entity, attrname, index, **kwargs):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
        """generates a consistent value for 'attrname'"""
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    96
        eschema = self.eschema
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    97
        attrtype = str(eschema.destination(attrname)).lower()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    98
        # Before calling generate_%s functions, try to find values domain
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    99
        if self.choice_func is not None:
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   100
            values_domain = self.choice_func(eschema, attrname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   101
            if values_domain is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   102
                return choice(values_domain)
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   103
        gen_func = getattr(self, 'generate_%s_%s' % (eschema, attrname),
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   104
                           getattr(self, 'generate_Any_%s' % attrname, None))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   105
        if gen_func is not None:
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   106
            return gen_func(entity, index, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   107
        # If no specific values domain, then generate a dummy value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   108
        gen_func = getattr(self, 'generate_%s' % (attrtype))
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   109
        return gen_func(entity, attrname, index, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   110
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   111
    def generate_string(self, entity, attrname, index, format=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   112
        """generates a consistent value for 'attrname' if it's a string"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   113
        # First try to get choices
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   114
        choosed = self.get_choice(entity, attrname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   115
        if choosed is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   116
            return choosed
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   117
        # All other case, generate a default string
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   118
        attrlength = get_max_length(self.eschema, attrname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   119
        num_len = numlen(index)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   120
        if num_len >= attrlength:
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   121
            ascii = self.eschema.rdef(attrname).internationalizable
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   122
            return ('&'+decompose_b26(index, ascii))[:attrlength]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   123
        # always use plain text when no format is specified
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   124
        attrprefix = attrname[:max(attrlength-num_len-1, 0)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   125
        if format == 'text/html':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   126
            value = u'<span>�%s<b>%d</b></span>' % (attrprefix, index)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   127
        elif format == 'text/rest':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   128
            value = u"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
title
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   130
-----
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   131
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   132
* %s
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   133
* %d
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   134
* �&
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
""" % (attrprefix, index)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
            value = u'�&%s%d' % (attrprefix, index)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   138
        return value[:attrlength]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   139
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   140
    def generate_password(self, entity, attrname, index):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
        """generates a consistent value for 'attrname' if it's a password"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
        return u'toto'
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   143
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   144
    def generate_integer(self, entity, attrname, index):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   145
        """generates a consistent value for 'attrname' if it's an integer"""
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   146
        return self._constrained_generate(entity, attrname, 0, 1, index)
9207
c99e97e9f566 [testing] add missing generate_bigint method to ValueGenerator. Closes #3059327
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8238
diff changeset
   147
    generate_int = generate_bigint = generate_integer
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   148
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   149
    def generate_float(self, entity, attrname, index):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   150
        """generates a consistent value for 'attrname' if it's a float"""
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   151
        return self._constrained_generate(entity, attrname, 0.0, 1.0, index)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   152
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   153
    def generate_decimal(self, entity, attrname, index):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   154
        """generates a consistent value for 'attrname' if it's a float"""
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   155
        return Decimal(str(self.generate_float(entity, attrname, index)))
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   156
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   157
    def generate_datetime(self, entity, attrname, index):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   158
        """generates a random date (format is 'yyyy-mm-dd HH:MM')"""
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   159
        base = datetime(randint(2000, 2004), randint(1, 12), randint(1, 28), 11, index%60)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   160
        return self._constrained_generate(entity, attrname, base, timedelta(hours=1), index)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   161
7166
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7152
diff changeset
   162
    generate_tzdatetime = generate_datetime # XXX implementation should add a timezone
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7152
diff changeset
   163
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   164
    def generate_date(self, entity, attrname, index):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   165
        """generates a random date (format is 'yyyy-mm-dd')"""
4651
f9cd35dece09 [devtools/fill] fix date generation and its test
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4513
diff changeset
   166
        base = date(randint(2000, 2010), 1, 1) + timedelta(randint(1, 365))
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   167
        return self._constrained_generate(entity, attrname, base, timedelta(days=1), index)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   168
6884
6fa712e9dfa5 [testlib] missing method on value generator to generate the Interval type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   169
    def generate_interval(self, entity, attrname, index):
6fa712e9dfa5 [testlib] missing method on value generator to generate the Interval type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   170
        """generates a random date (format is 'yyyy-mm-dd')"""
6fa712e9dfa5 [testlib] missing method on value generator to generate the Interval type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   171
        base = timedelta(randint(1, 365))
6fa712e9dfa5 [testlib] missing method on value generator to generate the Interval type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   172
        return self._constrained_generate(entity, attrname, base, timedelta(days=1), index)
6fa712e9dfa5 [testlib] missing method on value generator to generate the Interval type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   173
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   174
    def generate_time(self, entity, attrname, index):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   175
        """generates a random time (format is ' HH:MM')"""
2118
0012d997b992 should use time here, not timedelta
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   176
        return time(11, index%60) #'11:%02d' % (index % 60)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   177
7166
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7152
diff changeset
   178
    generate_tztime = generate_time # XXX implementation should add a timezone
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7152
diff changeset
   179
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   180
    def generate_bytes(self, entity, attrname, index, format=None):
10704
73367a56ee41 [py3k] ensure Binary objects are initialized with bytes
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10702
diff changeset
   181
        fakefile = Binary(("%s%s" % (attrname, index)).encode('ascii'))
3541
63ba0a669c4f unicode file name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   182
        fakefile.filename = u"file_%s" % attrname
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   183
        return fakefile
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   184
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   185
    def generate_boolean(self, entity, attrname, index):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   186
        """generates a consistent value for 'attrname' if it's a boolean"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   187
        return index % 2 == 0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   188
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   189
    def _constrained_generate(self, entity, attrname, base, step, index):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   190
        choosed = self.get_choice(entity, attrname)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   191
        if choosed is not None:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   192
            return choosed
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   193
        # ensure index > 0
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   194
        index += 1
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   195
        minvalue, maxvalue = self.get_bounds(entity, attrname)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   196
        if maxvalue is None:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   197
            if minvalue is not None:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   198
                base = max(minvalue, base)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   199
            maxvalue = base + index * step
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   200
        if minvalue is None:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   201
            minvalue = maxvalue - (index * step) # i.e. randint(-index, 0)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   202
        return choice(list(custom_range(minvalue, maxvalue, step)))
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   203
6916
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   204
    def _actual_boundary(self, entity, attrname, boundary):
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   205
        if isinstance(boundary, Attribute):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   206
            # ensure we've a value for this attribute
6916
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   207
            entity[attrname] = None # infinite loop safety belt
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   208
            if not boundary.attr in entity:
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   209
                self.generate_attribute_value(entity, boundary.attr)
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   210
            boundary = actual_value(boundary, entity)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   211
        return boundary
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   212
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   213
    def get_bounds(self, entity, attrname):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   214
        minvalue = maxvalue = None
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   215
        for cst in self.eschema.rdef(attrname).constraints:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   216
            if isinstance(cst, IntervalBoundConstraint):
6916
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   217
                minvalue = self._actual_boundary(entity, attrname, cst.minvalue)
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   218
                maxvalue = self._actual_boundary(entity, attrname, cst.maxvalue)
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   219
            elif isinstance(cst, BoundaryConstraint):
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   220
                if cst.operator[0] == '<':
6916
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   221
                    maxvalue = self._actual_boundary(entity, attrname, cst.boundary)
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   222
                else:
6916
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   223
                    minvalue = self._actual_boundary(entity, attrname, cst.boundary)
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   224
        return minvalue, maxvalue
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   225
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   226
    def get_choice(self, entity, attrname):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   227
        """generates a consistent value for 'attrname' if it has some static
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   228
        vocabulary set, else return None.
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   229
        """
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   230
        for cst in self.eschema.rdef(attrname).constraints:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   231
            if isinstance(cst, StaticVocabularyConstraint):
10702
f94c812c3669 [py3k] unicode → six.text_type
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10688
diff changeset
   232
                return text_type(choice(cst.vocabulary()))
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   233
        return None
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   234
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   235
    # XXX nothing to do here
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   236
    def generate_Any_data_format(self, entity, index, **kwargs):
5575
8a531340c3ef no more Image type w/ cw 3.9
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
   237
        # data_format attribute of File has no vocabulary constraint, we
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   238
        # need this method else stupid values will be set which make mtconverter
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   239
        # raise exception
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   240
        return u'application/octet-stream'
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   241
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   242
    def generate_Any_content_format(self, entity, index, **kwargs):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   243
        # content_format attribute of EmailPart has no vocabulary constraint, we
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   244
        # need this method else stupid values will be set which make mtconverter
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   245
        # raise exception
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   246
        return u'text/plain'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   247
7995
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   248
    def generate_CWDataImport_log(self, entity, index, **kwargs):
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   249
        # content_format attribute of EmailPart has no vocabulary constraint, we
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   250
        # need this method else stupid values will be set which make mtconverter
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   251
        # raise exception
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   252
        logs =  [u'%s\t%s\t%s\t%s<br/>' % (logging.ERROR, 'http://url.com?arg1=hop&arg2=hip',
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   253
                                           1, xml_escape('hjoio&oio"'))]
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   254
        return u'<br/>'.join(logs)
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   255
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   256
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   257
class autoextend(type):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   258
    def __new__(mcs, name, bases, classdict):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   259
        for attrname, attrvalue in classdict.items():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   260
            if callable(attrvalue):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   261
                if attrname.startswith('generate_') and \
10714
8a2ec43fcf44 [py3k] use inspect module to get a function's arguments
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10704
diff changeset
   262
                       len(inspect.getargspec(attrvalue).args) < 2:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   263
                    raise TypeError('generate_xxx must accept at least 1 argument')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   264
                setattr(_ValueGenerator, attrname, attrvalue)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   265
        return type.__new__(mcs, name, bases, classdict)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   266
10688
fa29f3628a1b [py3k] use six.add_metaclass
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10609
diff changeset
   267
fa29f3628a1b [py3k] use six.add_metaclass
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10609
diff changeset
   268
@add_metaclass(autoextend)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   269
class ValueGenerator(_ValueGenerator):
10688
fa29f3628a1b [py3k] use six.add_metaclass
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10609
diff changeset
   270
    pass
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   271
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   272
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   273
def _default_choice_func(etype, attrname):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   274
    """default choice_func for insert_entity_queries"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   275
    return None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   276
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   277
def insert_entity_queries(etype, schema, vreg, entity_num,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   278
                          choice_func=_default_choice_func):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   279
    """returns a list of 'add entity' queries (couples query, args)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   280
    :type etype: str
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   281
    :param etype: the entity's type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   282
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   283
    :type schema: cubicweb.schema.Schema
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2118
diff changeset
   284
    :param schema: the instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   285
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   286
    :type entity_num: int
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   287
    :param entity_num: the number of entities to insert
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   288
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   289
    XXX FIXME: choice_func is here for *historical* reasons, it should
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
               probably replaced by a nicer way to specify choices
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   291
    :type choice_func: function
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   292
    :param choice_func: a function that takes an entity type, an attrname and
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   293
                        returns acceptable values for this attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   294
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   295
    queries = []
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   296
    for index in range(entity_num):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   297
        restrictions = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   298
        args = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   299
        for attrname, value in make_entity(etype, schema, vreg, index, choice_func).items():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   300
            restrictions.append('X %s %%(%s)s' % (attrname, attrname))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   301
            args[attrname] = value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   302
        if restrictions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   303
            queries.append(('INSERT %s X: %s' % (etype, ', '.join(restrictions)),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   304
                            args))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   305
            assert not 'eid' in args, args
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   306
        else:
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   307
            queries.append(('INSERT %s X' % etype, {}))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   308
    return queries
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   309
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   310
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   311
def make_entity(etype, schema, vreg, index=0, choice_func=_default_choice_func,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   312
                form=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   313
    """generates a random entity and returns it as a dict
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   315
    by default, generate an entity to be inserted in the repository
8238
087bb529035c [spelling] fix dictionnary -> dictionary typo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7995
diff changeset
   316
    elif form, generate an form dictionary to be given to a web controller
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   317
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   318
    eschema = schema.eschema(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   319
    valgen = ValueGenerator(eschema, choice_func)
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   320
    entity = attrdict()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
    # preprocessing to deal with _format fields
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   322
    attributes = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
    relatedfields = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
    for rschema, attrschema in eschema.attribute_definitions():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
        attrname = rschema.type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
        if attrname == 'eid':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   327
            # don't specify eids !
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
            continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
        if attrname.endswith('_format') and attrname[:-7] in eschema.subject_relations():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   330
            relatedfields[attrname[:-7]] = attrschema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   332
            attributes.append((attrname, attrschema))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   333
    for attrname, attrschema in attributes:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   334
        if attrname in relatedfields:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   335
            # first generate a format and record it
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   336
            format = valgen.generate_attribute_value(entity, attrname + '_format', index)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   337
            # then a value coherent with this format
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   338
            value = valgen.generate_attribute_value(entity, attrname, index, format=format)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   339
        else:
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   340
            value = valgen.generate_attribute_value(entity, attrname, index)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   341
        if form: # need to encode values
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   342
            if attrschema.type == 'Bytes':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   343
                # twisted way
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   344
                fakefile = value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
                filename = value.filename
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   346
                value = (filename, u"text/plain", fakefile)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   347
            elif attrschema.type == 'Date':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   348
                value = value.strftime(vreg.property_value('ui.date-format'))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
            elif attrschema.type == 'Datetime':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   350
                value = value.strftime(vreg.property_value('ui.datetime-format'))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   351
            elif attrschema.type == 'Time':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   352
                value = value.strftime(vreg.property_value('ui.time-format'))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   353
            elif attrschema.type == 'Float':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   354
                fmt = vreg.property_value('ui.float-format')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   355
                value = fmt % value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   356
            else:
10702
f94c812c3669 [py3k] unicode → six.text_type
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10688
diff changeset
   357
                value = text_type(value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   358
    return entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   359
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   360
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   361
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
   362
def select(constraints, cnx, selectvar='O', objtype=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   363
    """returns list of eids matching <constraints>
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   364
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   365
    <selectvar> should be either 'O' or 'S' to match schema definitions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   366
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   367
    try:
3742
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   368
        rql = 'Any %s WHERE %s' % (selectvar, constraints)
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   369
        if objtype:
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   370
            rql += ', %s is %s' % (selectvar, objtype)
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
   371
        rset = cnx.execute(rql)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7445
diff changeset
   372
    except Exception:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9837
diff changeset
   373
        print("could restrict eid_list with given constraints (%r)" % constraints)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   374
        return []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   375
    return set(eid for eid, in rset.rows)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   376
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   377
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   378
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
   379
def make_relations_queries(schema, edict, cnx, ignored_relations=(),
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   380
                           existingrels=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   381
    """returns a list of generated RQL queries for relations
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2118
diff changeset
   382
    :param schema: The instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   383
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   384
    :param e_dict: mapping between etypes and eids
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   386
    :param ignored_relations: list of relations to ignore (i.e. don't try
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   387
                              to generate insert queries for these relations)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   388
    """
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
   389
    gen = RelationsQueriesGenerator(schema, cnx, existingrels)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   390
    return gen.compute_queries(edict, ignored_relations)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   391
3742
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   392
def composite_relation(rschema):
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   393
    for obj in rschema.objects():
9521
9eb810333b0f [hooks/security, devtools/fill] silence yams 0.38.0 warnings
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9207
diff changeset
   394
        if obj.rdef(rschema, 'object', takefirst=True).composite == 'subject':
3742
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   395
            return True
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   396
    for obj in rschema.subjects():
9521
9eb810333b0f [hooks/security, devtools/fill] silence yams 0.38.0 warnings
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9207
diff changeset
   397
        if obj.rdef(rschema, 'subject', takefirst=True).composite == 'object':
3742
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   398
            return True
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   399
    return False
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   400
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
class RelationsQueriesGenerator(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   402
    rql_tmpl = 'SET S %s O WHERE S eid %%(subjeid)s, O eid %%(objeid)s'
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
   403
    def __init__(self, schema, cnx, existing=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   404
        self.schema = schema
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
   405
        self.cnx = cnx
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   406
        self.existingrels = existing or {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   407
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   408
    def compute_queries(self, edict, ignored_relations):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   409
        queries = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   410
        #   1/ skip final relations and explictly ignored relations
3742
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   411
        rels = sorted([rschema for rschema in self.schema.relations()
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   412
                       if not (rschema.final or rschema in ignored_relations)],
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   413
                      key=lambda x:not composite_relation(x))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   414
        # for each relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   415
        #   2/ take each possible couple (subj, obj)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   416
        #   3/ analyze cardinality of relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
        #      a/ if relation is mandatory, insert one relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   418
        #      b/ else insert N relations where N is the mininum
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   419
        #         of 20 and the number of existing targetable entities
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   420
        for rschema in rels:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
            sym = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   422
            sedict = deepcopy(edict)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
            oedict = deepcopy(edict)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   424
            delayed = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   425
            # for each couple (subjschema, objschema), insert relations
4053
7cc66b1d9183 more api update
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 3749
diff changeset
   426
            for subj, obj in rschema.rdefs:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   427
                sym.add( (subj, obj) )
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4337
diff changeset
   428
                if rschema.symmetric and (obj, subj) in sym:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   429
                    continue
4053
7cc66b1d9183 more api update
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 3749
diff changeset
   430
                subjcard, objcard = rschema.rdef(subj, obj).cardinality
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   431
                # process mandatory relations first
3742
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   432
                if subjcard in '1+' or objcard in '1+' or composite_relation(rschema):
3749
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   433
                    for query, args in self.make_relation_queries(sedict, oedict,
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   434
                                                          rschema, subj, obj):
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   435
                        yield query, args
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   436
                else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   437
                    delayed.append( (subj, obj) )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   438
            for subj, obj in delayed:
3749
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   439
                for query, args in self.make_relation_queries(sedict, oedict, rschema,
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   440
                                                              subj, obj):
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   441
                    yield query, args
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   442
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   443
    def qargs(self, subjeids, objeids, subjcard, objcard, subjeid, objeid):
4513
8abf464d2ffe fix and begin to document autofill algorithm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4467
diff changeset
   444
        if subjcard in '?1+':
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   445
            subjeids.remove(subjeid)
4513
8abf464d2ffe fix and begin to document autofill algorithm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4467
diff changeset
   446
        if objcard in '?1+':
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   447
            objeids.remove(objeid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   448
        return {'subjeid' : subjeid, 'objeid' : objeid}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   449
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   450
    def make_relation_queries(self, sedict, oedict, rschema, subj, obj):
4053
7cc66b1d9183 more api update
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 3749
diff changeset
   451
        rdef = rschema.rdef(subj, obj)
7cc66b1d9183 more api update
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 3749
diff changeset
   452
        subjcard, objcard = rdef.cardinality
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   453
        subjeids = sedict.get(subj, frozenset())
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   454
        used = self.existingrels[rschema.type]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   455
        preexisting_subjrels = set(subj for subj, obj in used)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   456
        preexisting_objrels = set(obj for subj, obj in used)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   457
        # if there are constraints, only select appropriate objeids
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   458
        q = self.rql_tmpl % rschema.type
4053
7cc66b1d9183 more api update
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 3749
diff changeset
   459
        constraints = [c for c in rdef.constraints
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   460
                       if isinstance(c, RQLConstraint)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   461
        if constraints:
7152
39c1ffc7d93f [schema, refactoring] use RQLExpression as base class for RQL constraint classes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
   462
            restrictions = ', '.join(c.expression for c in constraints)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   463
            q += ', %s' % restrictions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   464
            # restrict object eids if possible
3711
486a29d66f2a work around a problem in constraint application
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 3689
diff changeset
   465
            # XXX the attempt to restrict below in completely wrong
486a29d66f2a work around a problem in constraint application
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 3689
diff changeset
   466
            # disabling it for now
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
   467
            objeids = select(restrictions, self.cnx, objtype=obj)
3712
4b8d7838d74d revert change, but there's something fishy here
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 3711
diff changeset
   468
        else:
4b8d7838d74d revert change, but there's something fishy here
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 3711
diff changeset
   469
            objeids = oedict.get(obj, frozenset())
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   470
        if subjcard in '?1' or objcard in '?1':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   471
            for subjeid, objeid in used:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   472
                if subjcard in '?1' and subjeid in subjeids:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   473
                    subjeids.remove(subjeid)
3673
9342e6783bd2 [testlib] hummm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3541
diff changeset
   474
                    # XXX why?
9342e6783bd2 [testlib] hummm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3541
diff changeset
   475
                    #if objeid in objeids:
9342e6783bd2 [testlib] hummm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3541
diff changeset
   476
                    #    objeids.remove(objeid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   477
                if objcard in '?1' and objeid in objeids:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   478
                    objeids.remove(objeid)
3673
9342e6783bd2 [testlib] hummm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3541
diff changeset
   479
                    # XXX why?
9342e6783bd2 [testlib] hummm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3541
diff changeset
   480
                    #if subjeid in subjeids:
9342e6783bd2 [testlib] hummm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3541
diff changeset
   481
                    #    subjeids.remove(subjeid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   482
        if not subjeids:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   483
            check_card_satisfied(objcard, objeids, subj, rschema, obj)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   484
            return
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   485
        if not objeids:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   486
            check_card_satisfied(subjcard, subjeids, subj, rschema, obj)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   487
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   488
        if subjcard in '?1+':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   489
            for subjeid in tuple(subjeids):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   490
                # do not insert relation if this entity already has a relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   491
                if subjeid in preexisting_subjrels:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   492
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   493
                objeid = choose_eid(objeids, subjeid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   494
                if objeid is None or (subjeid, objeid) in used:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   495
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   496
                yield q, self.qargs(subjeids, objeids, subjcard, objcard,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   497
                                    subjeid, objeid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   498
                used.add( (subjeid, objeid) )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   499
                if not objeids:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   500
                    check_card_satisfied(subjcard, subjeids, subj, rschema, obj)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   501
                    break
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   502
        elif objcard in '?1+':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   503
            for objeid in tuple(objeids):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   504
                # do not insert relation if this entity already has a relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   505
                if objeid in preexisting_objrels:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   506
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   507
                subjeid = choose_eid(subjeids, objeid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   508
                if subjeid is None or (subjeid, objeid) in used:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   509
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   510
                yield q, self.qargs(subjeids, objeids, subjcard, objcard,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   511
                                    subjeid, objeid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   512
                used.add( (subjeid, objeid) )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   513
                if not subjeids:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   514
                    check_card_satisfied(objcard, objeids, subj, rschema, obj)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   515
                    break
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   516
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   517
            # FIXME: 20 should be read from config
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   518
            subjeidsiter = [choice(tuple(subjeids)) for i in range(min(len(subjeids), 20))]
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   519
            objeidsiter = [choice(tuple(objeids)) for i in range(min(len(objeids), 20))]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   520
            for subjeid, objeid in zip(subjeidsiter, objeidsiter):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   521
                if subjeid != objeid and not (subjeid, objeid) in used:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   522
                    used.add( (subjeid, objeid) )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   523
                    yield q, self.qargs(subjeids, objeids, subjcard, objcard,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   524
                                        subjeid, objeid)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   525
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   526
def check_card_satisfied(card, remaining, subj, rschema, obj):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   527
    if card in '1+' and remaining:
3749
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   528
        raise Exception("can't satisfy cardinality %s for relation %s %s %s" %
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   529
                        (card, subj, rschema, obj))
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   530
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   531
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   532
def choose_eid(values, avoid):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   533
    values = tuple(values)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   534
    if len(values) == 1 and values[0] == avoid:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   535
        return None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   536
    objeid = choice(values)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   537
    while objeid == avoid: # avoid infinite recursion like in X comment X
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   538
        objeid = choice(values)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   539
    return objeid
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   540
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   541
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   542
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   543
# UTILITIES FUNCS ##############################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   544
def make_tel(num_tel):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   545
    """takes an integer, converts is as a string and inserts
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   546
    white spaces each 2 chars (french notation)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   547
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   548
    num_list = list(str(num_tel))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   549
    for index in (6, 4, 2):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   550
        num_list.insert(index, ' ')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   551
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   552
    return ''.join(num_list)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   553
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   554
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   555
def numlen(number):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   556
    """returns the number's length"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   557
    return len(str(number))