devtools/fill.py
author Julien Cristau <julien.cristau@logilab.fr>
Mon, 09 Nov 2015 15:22:43 +0100
changeset 10869 575982c948a9
parent 10714 8a2ec43fcf44
child 11079 eba98431f705
permissions -rw-r--r--
[dataimport] remove drop_index parameter from massive store "drop_index=False" also implied not dropping any constraints, in particular foreign keys, which meant any attempt to import entities would fail, because we only add metadata (the entities table) after the entity's insertion.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     1
# -*- coding: iso-8859-1 -*-
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
     2
# copyright 2003-2014 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
     3
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
     4
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
     5
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
     6
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
     7
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
     8
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
     9
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    10
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    11
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    12
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    13
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    14
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    15
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    16
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    17
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4651
diff changeset
    18
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
5786
7c16a4e7a592 [test] fix stupd name error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
    19
"""This modules defines func / methods for creating test repositories"""
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9837
diff changeset
    20
from __future__ import print_function
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
7995
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
    24
import logging
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
from random import randint, choice
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
from copy import deepcopy
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    27
from datetime import datetime, date, time, timedelta
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 428
diff changeset
    28
from decimal import Decimal
10714
8a2ec43fcf44 [py3k] use inspect module to get a function's arguments
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10704
diff changeset
    29
import inspect
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    30
10702
f94c812c3669 [py3k] unicode → six.text_type
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10688
diff changeset
    31
from six import text_type, add_metaclass
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    32
from six.moves import range
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    33
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    34
from logilab.common import attrdict
7995
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
    35
from logilab.mtconverter import xml_escape
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    36
from yams.constraints import (SizeConstraint, StaticVocabularyConstraint,
6916
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
    37
                              IntervalBoundConstraint, BoundaryConstraint,
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    38
                              Attribute, actual_value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    39
from rql.utils import decompose_b26 as base_decompose_b26
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    40
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    41
from cubicweb import Binary
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    42
from cubicweb.schema import RQLConstraint
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    43
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    44
def custom_range(start, stop, step):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    45
    while start < stop:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    46
        yield start
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    47
        start += step
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    48
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    49
def decompose_b26(index, ascii=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
    """return a letter (base-26) decomposition of index"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    51
    if ascii:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
        return base_decompose_b26(index)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    53
    return base_decompose_b26(index, u'�abcdefghijklmnopqrstuvwxyz')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    54
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    55
def get_max_length(eschema, attrname):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    56
    """returns the maximum length allowed for 'attrname'"""
4053
7cc66b1d9183 more api update
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 3749
diff changeset
    57
    for cst in eschema.rdef(attrname).constraints:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    58
        if isinstance(cst, SizeConstraint) and cst.max:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    59
            return cst.max
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    60
    return 300
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    61
    #raise AttributeError('No Size constraint on attribute "%s"' % attrname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    62
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
_GENERATED_VALUES = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    64
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    65
class _ValueGenerator(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    66
    """generates integers / dates / strings / etc. to fill a DB table"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    67
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    68
    def __init__(self, eschema, choice_func=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    69
        """<choice_func> is a function that returns a list of possible
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    70
        choices for a given entity type and an attribute name. It should
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    71
        looks like :
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    72
            def values_for(etype, attrname):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    73
                # some stuff ...
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    74
                return alist_of_acceptable_values # or None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    75
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    76
        self.choice_func = choice_func
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    77
        self.eschema = eschema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    78
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    79
    def generate_attribute_value(self, entity, attrname, index=1, **kwargs):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    80
        if attrname in entity:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    81
            return entity[attrname]
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    82
        eschema = self.eschema
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    83
        if not eschema.has_unique_values(attrname):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    84
            value = self.__generate_value(entity, attrname, index, **kwargs)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    85
        else:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    86
            value = self.__generate_value(entity, attrname, index, **kwargs)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    87
            while value in _GENERATED_VALUES.get((eschema, attrname), ()):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    88
                index += 1
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    89
                value = self.__generate_value(entity, attrname, index, **kwargs)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    90
            _GENERATED_VALUES.setdefault((eschema, attrname), set()).add(value)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    91
        entity[attrname] = value
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
        return value
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
    93
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    94
    def __generate_value(self, entity, attrname, index, **kwargs):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
        """generates a consistent value for 'attrname'"""
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    96
        eschema = self.eschema
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    97
        attrtype = str(eschema.destination(attrname)).lower()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    98
        # Before calling generate_%s functions, try to find values domain
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    99
        if self.choice_func is not None:
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   100
            values_domain = self.choice_func(eschema, attrname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   101
            if values_domain is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   102
                return choice(values_domain)
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   103
        gen_func = getattr(self, 'generate_%s_%s' % (eschema, attrname),
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   104
                           getattr(self, 'generate_Any_%s' % attrname, None))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   105
        if gen_func is not None:
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   106
            return gen_func(entity, index, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   107
        # If no specific values domain, then generate a dummy value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   108
        gen_func = getattr(self, 'generate_%s' % (attrtype))
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   109
        return gen_func(entity, attrname, index, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   110
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   111
    def generate_string(self, entity, attrname, index, format=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   112
        """generates a consistent value for 'attrname' if it's a string"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   113
        # First try to get choices
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   114
        choosed = self.get_choice(entity, attrname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   115
        if choosed is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   116
            return choosed
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   117
        # All other case, generate a default string
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   118
        attrlength = get_max_length(self.eschema, attrname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   119
        num_len = numlen(index)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   120
        if num_len >= attrlength:
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   121
            ascii = self.eschema.rdef(attrname).internationalizable
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   122
            return ('&'+decompose_b26(index, ascii))[:attrlength]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   123
        # always use plain text when no format is specified
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   124
        attrprefix = attrname[:max(attrlength-num_len-1, 0)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   125
        if format == 'text/html':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   126
            value = u'<span>�%s<b>%d</b></span>' % (attrprefix, index)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   127
        elif format == 'text/rest':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   128
            value = u"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
title
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   130
-----
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   131
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   132
* %s
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   133
* %d
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   134
* �&
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
""" % (attrprefix, index)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
            value = u'�&%s%d' % (attrprefix, index)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   138
        return value[:attrlength]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   139
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   140
    def generate_password(self, entity, attrname, index):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
        """generates a consistent value for 'attrname' if it's a password"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
        return u'toto'
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   143
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   144
    def generate_integer(self, entity, attrname, index):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   145
        """generates a consistent value for 'attrname' if it's an integer"""
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   146
        return self._constrained_generate(entity, attrname, 0, 1, index)
9207
c99e97e9f566 [testing] add missing generate_bigint method to ValueGenerator. Closes #3059327
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8238
diff changeset
   147
    generate_int = generate_bigint = generate_integer
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   148
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   149
    def generate_float(self, entity, attrname, index):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   150
        """generates a consistent value for 'attrname' if it's a float"""
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   151
        return self._constrained_generate(entity, attrname, 0.0, 1.0, index)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   152
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   153
    def generate_decimal(self, entity, attrname, index):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   154
        """generates a consistent value for 'attrname' if it's a float"""
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   155
        return Decimal(str(self.generate_float(entity, attrname, index)))
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   156
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   157
    def generate_datetime(self, entity, attrname, index):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   158
        """generates a random date (format is 'yyyy-mm-dd HH:MM')"""
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   159
        base = datetime(randint(2000, 2004), randint(1, 12), randint(1, 28), 11, index%60)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   160
        return self._constrained_generate(entity, attrname, base, timedelta(hours=1), index)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   161
7166
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7152
diff changeset
   162
    generate_tzdatetime = generate_datetime # XXX implementation should add a timezone
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7152
diff changeset
   163
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   164
    def generate_date(self, entity, attrname, index):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   165
        """generates a random date (format is 'yyyy-mm-dd')"""
4651
f9cd35dece09 [devtools/fill] fix date generation and its test
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4513
diff changeset
   166
        base = date(randint(2000, 2010), 1, 1) + timedelta(randint(1, 365))
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   167
        return self._constrained_generate(entity, attrname, base, timedelta(days=1), index)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   168
6884
6fa712e9dfa5 [testlib] missing method on value generator to generate the Interval type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   169
    def generate_interval(self, entity, attrname, index):
6fa712e9dfa5 [testlib] missing method on value generator to generate the Interval type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   170
        """generates a random date (format is 'yyyy-mm-dd')"""
6fa712e9dfa5 [testlib] missing method on value generator to generate the Interval type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   171
        base = timedelta(randint(1, 365))
6fa712e9dfa5 [testlib] missing method on value generator to generate the Interval type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   172
        return self._constrained_generate(entity, attrname, base, timedelta(days=1), index)
6fa712e9dfa5 [testlib] missing method on value generator to generate the Interval type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   173
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   174
    def generate_time(self, entity, attrname, index):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   175
        """generates a random time (format is ' HH:MM')"""
2118
0012d997b992 should use time here, not timedelta
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   176
        return time(11, index%60) #'11:%02d' % (index % 60)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   177
7166
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7152
diff changeset
   178
    generate_tztime = generate_time # XXX implementation should add a timezone
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7152
diff changeset
   179
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   180
    def generate_bytes(self, entity, attrname, index, format=None):
10704
73367a56ee41 [py3k] ensure Binary objects are initialized with bytes
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10702
diff changeset
   181
        fakefile = Binary(("%s%s" % (attrname, index)).encode('ascii'))
3541
63ba0a669c4f unicode file name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   182
        fakefile.filename = u"file_%s" % attrname
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   183
        return fakefile
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   184
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   185
    def generate_boolean(self, entity, attrname, index):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   186
        """generates a consistent value for 'attrname' if it's a boolean"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   187
        return index % 2 == 0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   188
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   189
    def _constrained_generate(self, entity, attrname, base, step, index):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   190
        choosed = self.get_choice(entity, attrname)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   191
        if choosed is not None:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   192
            return choosed
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   193
        # ensure index > 0
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   194
        index += 1
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   195
        minvalue, maxvalue = self.get_bounds(entity, attrname)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   196
        if maxvalue is None:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   197
            if minvalue is not None:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   198
                base = max(minvalue, base)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   199
            maxvalue = base + index * step
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   200
        if minvalue is None:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   201
            minvalue = maxvalue - (index * step) # i.e. randint(-index, 0)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   202
        return choice(list(custom_range(minvalue, maxvalue, step)))
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   203
6916
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   204
    def _actual_boundary(self, entity, attrname, boundary):
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   205
        if isinstance(boundary, Attribute):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   206
            # ensure we've a value for this attribute
6916
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   207
            entity[attrname] = None # infinite loop safety belt
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   208
            if not boundary.attr in entity:
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   209
                self.generate_attribute_value(entity, boundary.attr)
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   210
            boundary = actual_value(boundary, entity)
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   211
        return boundary
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   212
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   213
    def get_bounds(self, entity, attrname):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   214
        minvalue = maxvalue = None
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   215
        for cst in self.eschema.rdef(attrname).constraints:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   216
            if isinstance(cst, IntervalBoundConstraint):
6916
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   217
                minvalue = self._actual_boundary(entity, attrname, cst.minvalue)
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   218
                maxvalue = self._actual_boundary(entity, attrname, cst.maxvalue)
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   219
            elif isinstance(cst, BoundaryConstraint):
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   220
                if cst.operator[0] == '<':
6916
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   221
                    maxvalue = self._actual_boundary(entity, attrname, cst.boundary)
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   222
                else:
6916
3970185a8f8c [devtools] fix understanding of boundary constraint on db automatic population
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5811
diff changeset
   223
                    minvalue = self._actual_boundary(entity, attrname, cst.boundary)
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   224
        return minvalue, maxvalue
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   225
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   226
    def get_choice(self, entity, attrname):
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   227
        """generates a consistent value for 'attrname' if it has some static
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   228
        vocabulary set, else return None.
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   229
        """
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   230
        for cst in self.eschema.rdef(attrname).constraints:
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   231
            if isinstance(cst, StaticVocabularyConstraint):
10702
f94c812c3669 [py3k] unicode → six.text_type
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10688
diff changeset
   232
                return text_type(choice(cst.vocabulary()))
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   233
        return None
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   234
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   235
    # XXX nothing to do here
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   236
    def generate_Any_data_format(self, entity, index, **kwargs):
5575
8a531340c3ef no more Image type w/ cw 3.9
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
   237
        # data_format attribute of File has no vocabulary constraint, we
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   238
        # need this method else stupid values will be set which make mtconverter
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   239
        # raise exception
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   240
        return u'application/octet-stream'
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   241
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   242
    def generate_Any_content_format(self, entity, index, **kwargs):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   243
        # content_format attribute of EmailPart has no vocabulary constraint, we
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   244
        # need this method else stupid values will be set which make mtconverter
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   245
        # raise exception
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   246
        return u'text/plain'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   247
7995
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   248
    def generate_CWDataImport_log(self, entity, index, **kwargs):
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   249
        # content_format attribute of EmailPart has no vocabulary constraint, we
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   250
        # need this method else stupid values will be set which make mtconverter
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   251
        # raise exception
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   252
        logs =  [u'%s\t%s\t%s\t%s<br/>' % (logging.ERROR, 'http://url.com?arg1=hop&arg2=hip',
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   253
                                           1, xml_escape('hjoio&oio"'))]
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   254
        return u'<br/>'.join(logs)
9a9f35ef418c Record a log of datafeed source imports (closes #2026097)
Julien Cristau <julien.cristau@logilab.fr>
parents: 7815
diff changeset
   255
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   256
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   257
class autoextend(type):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   258
    def __new__(mcs, name, bases, classdict):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   259
        for attrname, attrvalue in classdict.items():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   260
            if callable(attrvalue):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   261
                if attrname.startswith('generate_') and \
10714
8a2ec43fcf44 [py3k] use inspect module to get a function's arguments
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10704
diff changeset
   262
                       len(inspect.getargspec(attrvalue).args) < 2:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   263
                    raise TypeError('generate_xxx must accept at least 1 argument')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   264
                setattr(_ValueGenerator, attrname, attrvalue)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   265
        return type.__new__(mcs, name, bases, classdict)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   266
10688
fa29f3628a1b [py3k] use six.add_metaclass
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10609
diff changeset
   267
fa29f3628a1b [py3k] use six.add_metaclass
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10609
diff changeset
   268
@add_metaclass(autoextend)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   269
class ValueGenerator(_ValueGenerator):
10688
fa29f3628a1b [py3k] use six.add_metaclass
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10609
diff changeset
   270
    pass
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   271
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   272
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   273
def _default_choice_func(etype, attrname):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   274
    """default choice_func for insert_entity_queries"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   275
    return None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   276
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   277
def insert_entity_queries(etype, schema, vreg, entity_num,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   278
                          choice_func=_default_choice_func):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   279
    """returns a list of 'add entity' queries (couples query, args)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   280
    :type etype: str
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   281
    :param etype: the entity's type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   282
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   283
    :type schema: cubicweb.schema.Schema
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2118
diff changeset
   284
    :param schema: the instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   285
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   286
    :type entity_num: int
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   287
    :param entity_num: the number of entities to insert
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   288
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   289
    XXX FIXME: choice_func is here for *historical* reasons, it should
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
               probably replaced by a nicer way to specify choices
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   291
    :type choice_func: function
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   292
    :param choice_func: a function that takes an entity type, an attrname and
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   293
                        returns acceptable values for this attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   294
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   295
    queries = []
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   296
    for index in range(entity_num):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   297
        restrictions = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   298
        args = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   299
        for attrname, value in make_entity(etype, schema, vreg, index, choice_func).items():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   300
            restrictions.append('X %s %%(%s)s' % (attrname, attrname))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   301
            args[attrname] = value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   302
        if restrictions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   303
            queries.append(('INSERT %s X: %s' % (etype, ', '.join(restrictions)),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   304
                            args))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   305
            assert not 'eid' in args, args
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   306
        else:
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   307
            queries.append(('INSERT %s X' % etype, {}))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   308
    return queries
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   309
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   310
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   311
def make_entity(etype, schema, vreg, index=0, choice_func=_default_choice_func,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   312
                form=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   313
    """generates a random entity and returns it as a dict
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   315
    by default, generate an entity to be inserted in the repository
8238
087bb529035c [spelling] fix dictionnary -> dictionary typo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7995
diff changeset
   316
    elif form, generate an form dictionary to be given to a web controller
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   317
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   318
    eschema = schema.eschema(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   319
    valgen = ValueGenerator(eschema, choice_func)
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   320
    entity = attrdict()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
    # preprocessing to deal with _format fields
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   322
    attributes = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
    relatedfields = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
    for rschema, attrschema in eschema.attribute_definitions():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
        attrname = rschema.type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
        if attrname == 'eid':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   327
            # don't specify eids !
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
            continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
        if attrname.endswith('_format') and attrname[:-7] in eschema.subject_relations():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   330
            relatedfields[attrname[:-7]] = attrschema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   332
            attributes.append((attrname, attrschema))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   333
    for attrname, attrschema in attributes:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   334
        if attrname in relatedfields:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   335
            # first generate a format and record it
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   336
            format = valgen.generate_attribute_value(entity, attrname + '_format', index)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   337
            # then a value coherent with this format
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   338
            value = valgen.generate_attribute_value(entity, attrname, index, format=format)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   339
        else:
4337
27ea69e2cfea refactor attributes'value generator to deal with yams special
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   340
            value = valgen.generate_attribute_value(entity, attrname, index)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   341
        if form: # need to encode values
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   342
            if attrschema.type == 'Bytes':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   343
                # twisted way
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   344
                fakefile = value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
                filename = value.filename
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   346
                value = (filename, u"text/plain", fakefile)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   347
            elif attrschema.type == 'Date':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   348
                value = value.strftime(vreg.property_value('ui.date-format'))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
            elif attrschema.type == 'Datetime':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   350
                value = value.strftime(vreg.property_value('ui.datetime-format'))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   351
            elif attrschema.type == 'Time':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   352
                value = value.strftime(vreg.property_value('ui.time-format'))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   353
            elif attrschema.type == 'Float':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   354
                fmt = vreg.property_value('ui.float-format')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   355
                value = fmt % value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   356
            else:
10702
f94c812c3669 [py3k] unicode → six.text_type
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10688
diff changeset
   357
                value = text_type(value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   358
    return entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   359
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   360
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   361
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
   362
def select(constraints, cnx, selectvar='O', objtype=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   363
    """returns list of eids matching <constraints>
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   364
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   365
    <selectvar> should be either 'O' or 'S' to match schema definitions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   366
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   367
    try:
3742
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   368
        rql = 'Any %s WHERE %s' % (selectvar, constraints)
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   369
        if objtype:
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   370
            rql += ', %s is %s' % (selectvar, objtype)
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
   371
        rset = cnx.execute(rql)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7445
diff changeset
   372
    except Exception:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9837
diff changeset
   373
        print("could restrict eid_list with given constraints (%r)" % constraints)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   374
        return []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   375
    return set(eid for eid, in rset.rows)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   376
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   377
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   378
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
   379
def make_relations_queries(schema, edict, cnx, ignored_relations=(),
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   380
                           existingrels=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   381
    """returns a list of generated RQL queries for relations
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2118
diff changeset
   382
    :param schema: The instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   383
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   384
    :param e_dict: mapping between etypes and eids
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   386
    :param ignored_relations: list of relations to ignore (i.e. don't try
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   387
                              to generate insert queries for these relations)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   388
    """
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
   389
    gen = RelationsQueriesGenerator(schema, cnx, existingrels)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   390
    return gen.compute_queries(edict, ignored_relations)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   391
3742
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   392
def composite_relation(rschema):
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   393
    for obj in rschema.objects():
9521
9eb810333b0f [hooks/security, devtools/fill] silence yams 0.38.0 warnings
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9207
diff changeset
   394
        if obj.rdef(rschema, 'object', takefirst=True).composite == 'subject':
3742
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   395
            return True
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   396
    for obj in rschema.subjects():
9521
9eb810333b0f [hooks/security, devtools/fill] silence yams 0.38.0 warnings
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9207
diff changeset
   397
        if obj.rdef(rschema, 'subject', takefirst=True).composite == 'object':
3742
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   398
            return True
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   399
    return False
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   400
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
class RelationsQueriesGenerator(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   402
    rql_tmpl = 'SET S %s O WHERE S eid %%(subjeid)s, O eid %%(objeid)s'
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
   403
    def __init__(self, schema, cnx, existing=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   404
        self.schema = schema
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
   405
        self.cnx = cnx
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   406
        self.existingrels = existing or {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   407
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   408
    def compute_queries(self, edict, ignored_relations):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   409
        queries = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   410
        #   1/ skip final relations and explictly ignored relations
3742
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   411
        rels = sorted([rschema for rschema in self.schema.relations()
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   412
                       if not (rschema.final or rschema in ignored_relations)],
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   413
                      key=lambda x:not composite_relation(x))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   414
        # for each relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   415
        #   2/ take each possible couple (subj, obj)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   416
        #   3/ analyze cardinality of relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
        #      a/ if relation is mandatory, insert one relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   418
        #      b/ else insert N relations where N is the mininum
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   419
        #         of 20 and the number of existing targetable entities
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   420
        for rschema in rels:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
            sym = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   422
            sedict = deepcopy(edict)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
            oedict = deepcopy(edict)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   424
            delayed = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   425
            # for each couple (subjschema, objschema), insert relations
4053
7cc66b1d9183 more api update
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 3749
diff changeset
   426
            for subj, obj in rschema.rdefs:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   427
                sym.add( (subj, obj) )
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4337
diff changeset
   428
                if rschema.symmetric and (obj, subj) in sym:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   429
                    continue
4053
7cc66b1d9183 more api update
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 3749
diff changeset
   430
                subjcard, objcard = rschema.rdef(subj, obj).cardinality
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   431
                # process mandatory relations first
3742
20f429eb5f46 kill separate attribute client-side handling #473636
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3712
diff changeset
   432
                if subjcard in '1+' or objcard in '1+' or composite_relation(rschema):
3749
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   433
                    for query, args in self.make_relation_queries(sedict, oedict,
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   434
                                                          rschema, subj, obj):
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   435
                        yield query, args
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   436
                else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   437
                    delayed.append( (subj, obj) )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   438
            for subj, obj in delayed:
3749
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   439
                for query, args in self.make_relation_queries(sedict, oedict, rschema,
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   440
                                                              subj, obj):
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   441
                    yield query, args
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   442
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   443
    def qargs(self, subjeids, objeids, subjcard, objcard, subjeid, objeid):
4513
8abf464d2ffe fix and begin to document autofill algorithm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4467
diff changeset
   444
        if subjcard in '?1+':
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   445
            subjeids.remove(subjeid)
4513
8abf464d2ffe fix and begin to document autofill algorithm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4467
diff changeset
   446
        if objcard in '?1+':
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   447
            objeids.remove(objeid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   448
        return {'subjeid' : subjeid, 'objeid' : objeid}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   449
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   450
    def make_relation_queries(self, sedict, oedict, rschema, subj, obj):
4053
7cc66b1d9183 more api update
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 3749
diff changeset
   451
        rdef = rschema.rdef(subj, obj)
7cc66b1d9183 more api update
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 3749
diff changeset
   452
        subjcard, objcard = rdef.cardinality
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   453
        subjeids = sedict.get(subj, frozenset())
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   454
        used = self.existingrels[rschema.type]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   455
        preexisting_subjrels = set(subj for subj, obj in used)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   456
        preexisting_objrels = set(obj for subj, obj in used)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   457
        # if there are constraints, only select appropriate objeids
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   458
        q = self.rql_tmpl % rschema.type
4053
7cc66b1d9183 more api update
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 3749
diff changeset
   459
        constraints = [c for c in rdef.constraints
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   460
                       if isinstance(c, RQLConstraint)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   461
        if constraints:
7152
39c1ffc7d93f [schema, refactoring] use RQLExpression as base class for RQL constraint classes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
   462
            restrictions = ', '.join(c.expression for c in constraints)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   463
            q += ', %s' % restrictions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   464
            # restrict object eids if possible
3711
486a29d66f2a work around a problem in constraint application
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 3689
diff changeset
   465
            # XXX the attempt to restrict below in completely wrong
486a29d66f2a work around a problem in constraint application
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 3689
diff changeset
   466
            # disabling it for now
9837
64c8ee99baf7 [devtools/testlib,fill] use the new connection api (for auto_populate)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9521
diff changeset
   467
            objeids = select(restrictions, self.cnx, objtype=obj)
3712
4b8d7838d74d revert change, but there's something fishy here
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 3711
diff changeset
   468
        else:
4b8d7838d74d revert change, but there's something fishy here
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 3711
diff changeset
   469
            objeids = oedict.get(obj, frozenset())
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   470
        if subjcard in '?1' or objcard in '?1':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   471
            for subjeid, objeid in used:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   472
                if subjcard in '?1' and subjeid in subjeids:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   473
                    subjeids.remove(subjeid)
3673
9342e6783bd2 [testlib] hummm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3541
diff changeset
   474
                    # XXX why?
9342e6783bd2 [testlib] hummm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3541
diff changeset
   475
                    #if objeid in objeids:
9342e6783bd2 [testlib] hummm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3541
diff changeset
   476
                    #    objeids.remove(objeid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   477
                if objcard in '?1' and objeid in objeids:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   478
                    objeids.remove(objeid)
3673
9342e6783bd2 [testlib] hummm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3541
diff changeset
   479
                    # XXX why?
9342e6783bd2 [testlib] hummm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3541
diff changeset
   480
                    #if subjeid in subjeids:
9342e6783bd2 [testlib] hummm
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3541
diff changeset
   481
                    #    subjeids.remove(subjeid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   482
        if not subjeids:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   483
            check_card_satisfied(objcard, objeids, subj, rschema, obj)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   484
            return
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   485
        if not objeids:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   486
            check_card_satisfied(subjcard, subjeids, subj, rschema, obj)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   487
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   488
        if subjcard in '?1+':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   489
            for subjeid in tuple(subjeids):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   490
                # do not insert relation if this entity already has a relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   491
                if subjeid in preexisting_subjrels:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   492
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   493
                objeid = choose_eid(objeids, subjeid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   494
                if objeid is None or (subjeid, objeid) in used:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   495
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   496
                yield q, self.qargs(subjeids, objeids, subjcard, objcard,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   497
                                    subjeid, objeid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   498
                used.add( (subjeid, objeid) )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   499
                if not objeids:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   500
                    check_card_satisfied(subjcard, subjeids, subj, rschema, obj)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   501
                    break
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   502
        elif objcard in '?1+':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   503
            for objeid in tuple(objeids):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   504
                # do not insert relation if this entity already has a relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   505
                if objeid in preexisting_objrels:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   506
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   507
                subjeid = choose_eid(subjeids, objeid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   508
                if subjeid is None or (subjeid, objeid) in used:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   509
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   510
                yield q, self.qargs(subjeids, objeids, subjcard, objcard,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   511
                                    subjeid, objeid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   512
                used.add( (subjeid, objeid) )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   513
                if not subjeids:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   514
                    check_card_satisfied(objcard, objeids, subj, rschema, obj)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   515
                    break
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   516
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   517
            # FIXME: 20 should be read from config
10609
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   518
            subjeidsiter = [choice(tuple(subjeids)) for i in range(min(len(subjeids), 20))]
e2d8e81bfe68 [py3k] import range using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   519
            objeidsiter = [choice(tuple(objeids)) for i in range(min(len(objeids), 20))]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   520
            for subjeid, objeid in zip(subjeidsiter, objeidsiter):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   521
                if subjeid != objeid and not (subjeid, objeid) in used:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   522
                    used.add( (subjeid, objeid) )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   523
                    yield q, self.qargs(subjeids, objeids, subjcard, objcard,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   524
                                        subjeid, objeid)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   525
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   526
def check_card_satisfied(card, remaining, subj, rschema, obj):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   527
    if card in '1+' and remaining:
3749
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   528
        raise Exception("can't satisfy cardinality %s for relation %s %s %s" %
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   529
                        (card, subj, rschema, obj))
a84e798b99f6 turn make_relations_query into a true generator (this allows interleaving of actual entities linking and query generation)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3747
diff changeset
   530
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   531
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   532
def choose_eid(values, avoid):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   533
    values = tuple(values)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   534
    if len(values) == 1 and values[0] == avoid:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   535
        return None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   536
    objeid = choice(values)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   537
    while objeid == avoid: # avoid infinite recursion like in X comment X
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   538
        objeid = choice(values)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   539
    return objeid
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   540
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1398
diff changeset
   541
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   542
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   543
# UTILITIES FUNCS ##############################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   544
def make_tel(num_tel):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   545
    """takes an integer, converts is as a string and inserts
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   546
    white spaces each 2 chars (french notation)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   547
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   548
    num_list = list(str(num_tel))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   549
    for index in (6, 4, 2):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   550
        num_list.insert(index, ' ')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   551
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   552
    return ''.join(num_list)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   553
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   554
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   555
def numlen(number):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   556
    """returns the number's length"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   557
    return len(str(number))