cubicweb/dataimport/stores.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Mon, 01 Feb 2016 17:31:35 +0100
changeset 11309 31bf3254be69
parent 11308 df75fe529ba8
child 11749 ae9789d77ea0
permissions -rw-r--r--
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator with a similar-but-different API that will allow its usage by the massive store (no more entity object manipulation), while making it a bit more flexible for customization by subclasses (experience taken from client code). Related to #10338621
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
11140
fabcd1c6dcd1 [dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11057
diff changeset
     1
# copyright 2003-2016 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     3
#
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     4
# This file is part of CubicWeb.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     5
#
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     9
# any later version.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    10
#
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    14
# details.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    15
#
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    18
"""
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    19
Stores are responsible to insert properly formatted entities and relations into the database. They
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    20
have the following API::
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    21
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    22
    >>> user_eid = store.prepare_insert_entity('CWUser', login=u'johndoe')
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    23
    >>> group_eid = store.prepare_insert_entity('CWUser', name=u'unknown')
10794
0b4d6beec132 [dataimport] Update doc to reflect new API
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10662
diff changeset
    24
    >>> store.prepare_insert_relation(user_eid, 'in_group', group_eid)
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    25
    >>> store.flush()
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    26
    >>> store.commit()
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    27
    >>> store.finish()
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    28
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    29
Some store **requires a flush** to copy data in the database, so if you want to have store
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    30
independant code you should explicitly call it. (There may be multiple flushes during the
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    31
process, or only one at the end if there is no memory issue). This is different from the
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    32
commit which validates the database transaction. At last, the `finish()` method should be called in
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    33
case the store requires additional work once everything is done.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    34
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    35
* ``prepare_insert_entity(<entity type>, **kwargs) -> eid``: given an entity
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    36
  type, attributes and inlined relations, return the eid of the entity to be
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    37
  inserted, *with no guarantee that anything has been inserted in database*,
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    38
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    39
* ``prepare_update_entity(<entity type>, eid, **kwargs) -> None``: given an
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    40
  entity type and eid, promise for update given attributes and inlined
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    41
  relations *with no guarantee that anything has been inserted in database*,
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    42
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    43
* ``prepare_insert_relation(eid_from, rtype, eid_to) -> None``: indicate that a
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    44
  relation ``rtype`` should be added between entities with eids ``eid_from``
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    45
  and ``eid_to``. Similar to ``prepare_insert_entity()``, *there is no
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    46
  guarantee that the relation will be inserted in database*,
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    47
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    48
* ``flush() -> None``: flush any temporary data to database. May be called
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    49
  several times during an import,
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    50
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    51
* ``commit() -> None``: commit the database transaction,
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    52
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    53
* ``finish() -> None``: additional stuff to do after import is terminated.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    54
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    55
.. autoclass:: cubicweb.dataimport.stores.RQLObjectStore
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    56
.. autoclass:: cubicweb.dataimport.stores.NoHookRQLObjectStore
11309
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
    57
.. autoclass:: cubicweb.dataimport.stores.MetadataGenerator
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    58
"""
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    59
import inspect
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    60
import warnings
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    61
from datetime import datetime
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    62
from copy import copy
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    63
11309
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
    64
from six import text_type, add_metaclass
10809
359cbdf3a515 [dataimport] extid must be a bytes object
Julien Cristau <julien.cristau@logilab.fr>
parents: 10794
diff changeset
    65
11308
df75fe529ba8 [dataimport] Prefer now(pytz.utc) to utcnow
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11307
diff changeset
    66
import pytz
df75fe529ba8 [dataimport] Prefer now(pytz.utc) to utcnow
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11307
diff changeset
    67
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    68
from logilab.common.decorators import cached
11309
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
    69
from logilab.common.deprecation import deprecated, class_deprecated
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    70
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    71
from cubicweb.schema import META_RTYPES, VIRTUAL_RTYPES
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    72
from cubicweb.server.edition import EditedEntity
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    73
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    74
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    75
class RQLObjectStore(object):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    76
    """Store that works by making RQL queries, hence with all the cubicweb's machinery activated.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    77
    """
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    78
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    79
    def __init__(self, cnx, commit=None):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    80
        if commit is not None:
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    81
            warnings.warn('[3.19] commit argument should not be specified '
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    82
                          'as the cnx object already provides it.',
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    83
                          DeprecationWarning, stacklevel=2)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    84
        self._cnx = cnx
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    85
        self._commit = commit or cnx.commit
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    86
        # XXX 3.21 deprecated attributes
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    87
        self.eids = {}
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    88
        self.types = {}
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    89
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    90
    def rql(self, *args):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    91
        """Execute a RQL query. This is NOT part of the store API."""
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    92
        return self._cnx.execute(*args)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    93
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    94
    def prepare_insert_entity(self, *args, **kwargs):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    95
        """Given an entity type, attributes and inlined relations, returns the inserted entity's
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    96
        eid.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    97
        """
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    98
        entity = self._cnx.create_entity(*args, **kwargs)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    99
        self.eids[entity.eid] = entity
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   100
        self.types.setdefault(args[0], []).append(entity.eid)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   101
        return entity.eid
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   102
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   103
    def prepare_update_entity(self, etype, eid, **kwargs):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   104
        """Given an entity type and eid, updates the corresponding entity with specified attributes
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   105
        and inlined relations.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   106
        """
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   107
        entity = self._cnx.entity_from_eid(eid)
10939
b30c2f49da57 [dataimport] Format strings with % instead of .format()
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 10907
diff changeset
   108
        assert entity.cw_etype == etype, 'Trying to update with wrong type %s' % etype
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   109
        # XXX some inlined relations may already exists
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   110
        entity.cw_set(**kwargs)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   111
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   112
    def prepare_insert_relation(self, eid_from, rtype, eid_to, **kwargs):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   113
        """Insert into the database a  relation ``rtype`` between entities with eids ``eid_from``
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   114
        and ``eid_to``.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   115
        """
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   116
        self.rql('SET X %s Y WHERE X eid %%(x)s, Y eid %%(y)s' % rtype,
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   117
                 {'x': int(eid_from), 'y': int(eid_to)})
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   118
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   119
    def flush(self):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   120
        """Nothing to flush for this store."""
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   121
        pass
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   122
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   123
    def commit(self):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   124
        """Commit the database transaction."""
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   125
        return self._commit()
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   126
10862
7357b1485795 [dataimport] .finish method was missing from (NoHook)RQLObjectStore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10809
diff changeset
   127
    def finish(self):
7357b1485795 [dataimport] .finish method was missing from (NoHook)RQLObjectStore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10809
diff changeset
   128
        """Nothing to do once import is terminated for this store."""
7357b1485795 [dataimport] .finish method was missing from (NoHook)RQLObjectStore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10809
diff changeset
   129
        pass
7357b1485795 [dataimport] .finish method was missing from (NoHook)RQLObjectStore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10809
diff changeset
   130
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   131
    @property
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   132
    def session(self):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   133
        warnings.warn('[3.19] deprecated property.', DeprecationWarning, stacklevel=2)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   134
        return self._cnx.repo._get_session(self._cnx.sessionid)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   135
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   136
    @deprecated("[3.19] use cnx.find(*args, **kwargs).entities() instead")
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   137
    def find_entities(self, *args, **kwargs):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   138
        return self._cnx.find(*args, **kwargs).entities()
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   139
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   140
    @deprecated("[3.19] use cnx.find(*args, **kwargs).one() instead")
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   141
    def find_one_entity(self, *args, **kwargs):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   142
        return self._cnx.find(*args, **kwargs).one()
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   143
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   144
    @deprecated('[3.21] use prepare_insert_entity instead')
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   145
    def create_entity(self, *args, **kwargs):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   146
        eid = self.prepare_insert_entity(*args, **kwargs)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   147
        return self._cnx.entity_from_eid(eid)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   148
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   149
    @deprecated('[3.21] use prepare_insert_relation instead')
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   150
    def relate(self, eid_from, rtype, eid_to, **kwargs):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   151
        self.prepare_insert_relation(eid_from, rtype, eid_to, **kwargs)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   152
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   153
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   154
class NoHookRQLObjectStore(RQLObjectStore):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   155
    """Store that works by accessing low-level CubicWeb's source API, with all hooks deactivated. It
11307
74f5814ecdf0 [dataimport] rename nohook store .source attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11304
diff changeset
   156
    may be given a metadata generator object to handle metadata which are usually handled by hooks.
74f5814ecdf0 [dataimport] rename nohook store .source attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11304
diff changeset
   157
74f5814ecdf0 [dataimport] rename nohook store .source attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11304
diff changeset
   158
    Arguments:
74f5814ecdf0 [dataimport] rename nohook store .source attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11304
diff changeset
   159
    - `cnx`, a connection to the repository
11309
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   160
    - `metagen`, optional :class:`MetadataGenerator` instance
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   161
    """
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   162
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   163
    def __init__(self, cnx, metagen=None):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   164
        super(NoHookRQLObjectStore, self).__init__(cnx)
11309
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   165
        if metagen is None:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   166
            metagen = MetadataGenerator(cnx)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   167
        if isinstance(metagen, MetadataGenerator):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   168
            metagen = _MetaGeneratorBWCompatWrapper(metagen)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   169
        self.metagen = metagen
11307
74f5814ecdf0 [dataimport] rename nohook store .source attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11304
diff changeset
   170
        self._system_source = cnx.repo.system_source
11140
fabcd1c6dcd1 [dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11057
diff changeset
   171
        self._rschema = cnx.repo.schema.rschema
11309
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   172
        self._create_eid = self._system_source.create_eid
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   173
        self._add_relation = self._system_source.add_relation
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   174
        self._nb_inserted_entities = 0
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   175
        self._nb_inserted_types = 0
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   176
        self._nb_inserted_relations = 0
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   177
        # deactivate security
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   178
        cnx.read_security = False
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   179
        cnx.write_security = False
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   180
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   181
    def prepare_insert_entity(self, etype, **kwargs):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   182
        """Given an entity type, attributes and inlined relations, returns the inserted entity's
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   183
        eid.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   184
        """
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10513
diff changeset
   185
        for k, v in kwargs.items():
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   186
            kwargs[k] = getattr(v, 'eid', v)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   187
        entity, rels = self.metagen.base_etype_dicts(etype)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   188
        # make a copy to keep cached entity pristine
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   189
        entity = copy(entity)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   190
        entity.cw_edited = copy(entity.cw_edited)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   191
        entity.cw_clear_relation_cache()
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   192
        entity.cw_edited.update(kwargs, skipsec=False)
11304
1dad60d54dfb [dataimport] move eid attribution from the metagenerator back to the nohook store
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11303
diff changeset
   193
        cnx = self._cnx
1dad60d54dfb [dataimport] move eid attribution from the metagenerator back to the nohook store
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11303
diff changeset
   194
        entity.eid = self._create_eid(cnx)
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   195
        entity_source, extid = self.metagen.init_entity(entity)
11307
74f5814ecdf0 [dataimport] rename nohook store .source attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11304
diff changeset
   196
        self._system_source.add_info(cnx, entity, entity_source, extid)
74f5814ecdf0 [dataimport] rename nohook store .source attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11304
diff changeset
   197
        self._system_source.add_entity(cnx, entity)
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   198
        kwargs = dict()
11140
fabcd1c6dcd1 [dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11057
diff changeset
   199
        if inspect.getargspec(self._add_relation).keywords:
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   200
            kwargs['subjtype'] = entity.cw_etype
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10513
diff changeset
   201
        for rtype, targeteids in rels.items():
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   202
            # targeteids may be a single eid or a list of eids
11140
fabcd1c6dcd1 [dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11057
diff changeset
   203
            inlined = self._rschema(rtype).inlined
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   204
            try:
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   205
                for targeteid in targeteids:
11140
fabcd1c6dcd1 [dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11057
diff changeset
   206
                    self._add_relation(cnx, entity.eid, rtype, targeteid,
fabcd1c6dcd1 [dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11057
diff changeset
   207
                                       inlined, **kwargs)
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   208
            except TypeError:
11140
fabcd1c6dcd1 [dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11057
diff changeset
   209
                self._add_relation(cnx, entity.eid, rtype, targeteids,
fabcd1c6dcd1 [dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11057
diff changeset
   210
                                   inlined, **kwargs)
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   211
        self._nb_inserted_entities += 1
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   212
        return entity.eid
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   213
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   214
    # XXX: prepare_update_entity is inherited from RQLObjectStore, it should be reimplemented to
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   215
    # actually skip hooks as prepare_insert_entity
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   216
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   217
    def prepare_insert_relation(self, eid_from, rtype, eid_to, **kwargs):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   218
        """Insert into the database a  relation ``rtype`` between entities with eids ``eid_from``
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   219
        and ``eid_to``.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   220
        """
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   221
        assert not rtype.startswith('reverse_')
11140
fabcd1c6dcd1 [dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11057
diff changeset
   222
        rschema = self._rschema(rtype)
fabcd1c6dcd1 [dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11057
diff changeset
   223
        self._add_relation(self._cnx, eid_from, rtype, eid_to, rschema.inlined)
fabcd1c6dcd1 [dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11057
diff changeset
   224
        if rschema.symmetric:
fabcd1c6dcd1 [dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11057
diff changeset
   225
            self._add_relation(self._cnx, eid_to, rtype, eid_from, rschema.inlined)
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   226
        self._nb_inserted_relations += 1
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   227
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   228
    @property
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   229
    @deprecated('[3.21] deprecated')
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   230
    def nb_inserted_entities(self):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   231
        return self._nb_inserted_entities
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   232
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   233
    @property
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   234
    @deprecated('[3.21] deprecated')
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   235
    def nb_inserted_types(self):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   236
        return self._nb_inserted_types
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   237
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   238
    @property
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   239
    @deprecated('[3.21] deprecated')
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   240
    def nb_inserted_relations(self):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   241
        return self._nb_inserted_relations
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   242
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   243
11309
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   244
class MetadataGenerator(object):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   245
    """Class responsible for generating standard metadata for imported entities. You may want to
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   246
    derive it to add application specific's metadata. This class (or a subclass) may either be
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   247
    given to a nohook or massive store.
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   248
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   249
    Parameters:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   250
    * `cnx`: connection to the repository
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   251
    * `baseurl`: optional base URL to be used for `cwuri` generation - default to config['base-url']
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   252
    * `source`: optional source to be used as `cw_source` for imported entities
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   253
    """
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   254
    META_RELATIONS = (META_RTYPES
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   255
                      - VIRTUAL_RTYPES
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   256
                      - set(('eid', 'cwuri',
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   257
                             'is', 'is_instance_of', 'cw_source')))
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   258
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   259
    def __init__(self, cnx, baseurl=None, source=None):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   260
        self._cnx = cnx
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   261
        if baseurl is None:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   262
            config = cnx.vreg.config
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   263
            baseurl = config['base-url'] or config.default_base_url()
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   264
        if not baseurl[-1] == '/':
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   265
            baseurl += '/'
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   266
        self._baseurl = baseurl
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   267
        if source is None:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   268
            source = cnx.repo.system_source
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   269
        self.source = source
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   270
        self._need_extid = source is not cnx.repo.system_source
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   271
        self._now = datetime.now(pytz.utc)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   272
        # attributes/relations shared by all entities of the same type
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   273
        self._etype_attrs = []
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   274
        self._etype_rels = []
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   275
        # attributes/relations specific to each entity
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   276
        self._entity_attrs = ['cwuri']
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   277
        rschema = cnx.vreg.schema.rschema
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   278
        for rtype in self.META_RELATIONS:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   279
            # skip owned_by / created_by if user is the internal manager
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   280
            if cnx.user.eid == -1 and rtype in ('owned_by', 'created_by'):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   281
                continue
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   282
            if rschema(rtype).final:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   283
                self._etype_attrs.append(rtype)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   284
            else:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   285
                self._etype_rels.append(rtype)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   286
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   287
    # etype is provided in the 3 methods below as proven useful to custom implementation but not
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   288
    # used by the default implementation
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   289
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   290
    def etype_attrs(self, etype):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   291
        """Return the list of attributes to be set for all entities of the given type."""
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   292
        return self._etype_attrs[:]
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   293
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   294
    def etype_rels(self, etype):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   295
        """Return the list of relations to be set for all entities of the given type."""
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   296
        return self._etype_rels[:]
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   297
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   298
    def entity_attrs(self, etype):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   299
        """Return the list of attributes whose value is set per instance, not per type, for the
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   300
        given type.
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   301
        """
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   302
        return self._entity_attrs[:]
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   303
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   304
    @cached
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   305
    def base_etype_attrs(self, etype):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   306
        """Return a dictionary of attributes to be set for all entities of the given type."""
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   307
        attrs = {}
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   308
        for attr in self.etype_attrs(etype):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   309
            genfunc = self._generator(attr)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   310
            if genfunc:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   311
                attrs[attr] = genfunc(etype)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   312
        return attrs
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   313
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   314
    @cached
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   315
    def base_etype_rels(self, etype):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   316
        """Return a dictionary of relations to be set for all entities of the given type."""
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   317
        rels = {}
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   318
        for rel in self.etype_rels(etype):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   319
            genfunc = self._generator(rel)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   320
            if genfunc:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   321
                rels[rel] = genfunc(etype)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   322
        return rels
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   323
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   324
    def entity_extid(self, etype, eid, attrs):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   325
        """Return the extid for the entity of given type and eid, to be inserted in the 'entities'
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   326
        system table.
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   327
        """
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   328
        if self._need_extid:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   329
            extid = attrs.get('cwuri')
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   330
            if extid is None:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   331
                raise Exception('entity from an external source but no extid specified')
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   332
            elif isinstance(extid, text_type):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   333
                extid = extid.encode('utf-8')
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   334
        else:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   335
            extid = None
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   336
        return extid
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   337
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   338
    def init_entity_attrs(self, etype, eid, attrs):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   339
        """Insert into an entity attrs dictionary attributes whose value is set per instance, not per
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   340
        type.
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   341
        """
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   342
        for attr in self.entity_attrs(etype):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   343
            if attr in attrs:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   344
                # already set, skip this attribute
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   345
                continue
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   346
            genfunc = self._generator(attr)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   347
            if genfunc:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   348
                attrs[attr] = genfunc(etype, eid, attrs)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   349
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   350
    def _generator(self, rtype):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   351
        return getattr(self, 'gen_%s' % rtype, None)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   352
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   353
    def gen_cwuri(self, etype, eid, attrs):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   354
        assert self._baseurl, 'baseurl is None while generating cwuri'
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   355
        return u'%s%s' % (self._baseurl, eid)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   356
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   357
    def gen_creation_date(self, etype):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   358
        return self._now
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   359
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   360
    def gen_modification_date(self, etype):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   361
        return self._now
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   362
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   363
    def gen_created_by(self, etype):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   364
        return self._cnx.user.eid
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   365
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   366
    def gen_owned_by(self, etype):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   367
        return self._cnx.user.eid
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   368
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   369
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   370
class _MetaGeneratorBWCompatWrapper(object):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   371
    """Class wrapping a MetadataGenerator to adapt it to the MetaGenerator interface.
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   372
    """
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   373
    META_RELATIONS = (META_RTYPES
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   374
                      - VIRTUAL_RTYPES
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   375
                      - set(('eid', 'cwuri',
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   376
                             'is', 'is_instance_of', 'cw_source')))
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   377
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   378
    def __init__(self, mdgenerator):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   379
        self._mdgen = mdgenerator
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   380
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   381
    @cached
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   382
    def base_etype_dicts(self, etype):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   383
        cnx = self._mdgen._cnx
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   384
        entity = cnx.vreg['etypes'].etype_class(etype)(cnx)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   385
        # entity are "surface" copied, avoid shared dict between copies
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   386
        del entity.cw_extra_kwargs
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   387
        entity.cw_edited = EditedEntity(entity)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   388
        attrs = self._mdgen.base_etype_attrs(etype)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   389
        entity.cw_edited.update(attrs, skipsec=False)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   390
        rels = self._mdgen.base_etype_rels(etype)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   391
        return entity, rels
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   392
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   393
    def init_entity(self, entity):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   394
        # if cwuri is specified, this is an extid. It's not if it's generated in the above loop
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   395
        extid = self._mdgen.entity_extid(entity.cw_etype, entity.eid, entity.cw_edited)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   396
        attrs = dict(entity.cw_edited)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   397
        self._mdgen.init_entity_attrs(entity.cw_etype, entity.eid, attrs)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   398
        entity.cw_edited.update(attrs, skipsec=False)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   399
        return self._mdgen.source, extid
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   400
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   401
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   402
@add_metaclass(class_deprecated)
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   403
class MetaGenerator(object):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   404
    """Class responsible for generating standard metadata for imported entities. You may want to
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   405
    derive it to add application specific's metadata.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   406
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   407
    Parameters:
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   408
    * `cnx`: connection to the repository
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   409
    * `baseurl`: optional base URL to be used for `cwuri` generation - default to config['base-url']
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   410
    * `source`: optional source to be used as `cw_source` for imported entities
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   411
    """
11309
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   412
    __deprecation_warning__ = '[3.23] this class is deprecated, use MetadataGenerator instead'
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   413
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   414
    META_RELATIONS = (META_RTYPES
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   415
                      - VIRTUAL_RTYPES
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   416
                      - set(('eid', 'cwuri',
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   417
                             'is', 'is_instance_of', 'cw_source')))
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   418
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   419
    def __init__(self, cnx, baseurl=None, source=None):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   420
        self._cnx = cnx
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   421
        if baseurl is None:
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   422
            config = cnx.vreg.config
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   423
            baseurl = config['base-url'] or config.default_base_url()
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   424
        if not baseurl[-1] == '/':
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   425
            baseurl += '/'
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   426
        self.baseurl = baseurl
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   427
        if source is None:
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   428
            source = cnx.repo.system_source
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   429
        self.source = source
11308
df75fe529ba8 [dataimport] Prefer now(pytz.utc) to utcnow
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11307
diff changeset
   430
        self._now = datetime.now(pytz.utc)
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   431
        # attributes/relations shared by all entities of the same type
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   432
        self.etype_attrs = []
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   433
        self.etype_rels = []
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   434
        # attributes/relations specific to each entity
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   435
        self.entity_attrs = ['cwuri']
11303
891f3ecdbf7f [dataimport] rephrase minor bits of the meta generator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11142
diff changeset
   436
        rschema = cnx.vreg.schema.rschema
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   437
        for rtype in self.META_RELATIONS:
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   438
            # skip owned_by / created_by if user is the internal manager
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   439
            if cnx.user.eid == -1 and rtype in ('owned_by', 'created_by'):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   440
                continue
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   441
            if rschema(rtype).final:
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   442
                self.etype_attrs.append(rtype)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   443
            else:
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   444
                self.etype_rels.append(rtype)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   445
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   446
    @cached
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   447
    def base_etype_dicts(self, etype):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   448
        entity = self._cnx.vreg['etypes'].etype_class(etype)(self._cnx)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   449
        # entity are "surface" copied, avoid shared dict between copies
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   450
        del entity.cw_extra_kwargs
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   451
        entity.cw_edited = EditedEntity(entity)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   452
        for attr in self.etype_attrs:
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   453
            genfunc = self.generate(attr)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   454
            if genfunc:
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   455
                entity.cw_edited.edited_attribute(attr, genfunc(entity))
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   456
        rels = {}
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   457
        for rel in self.etype_rels:
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   458
            genfunc = self.generate(rel)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   459
            if genfunc:
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   460
                rels[rel] = genfunc(entity)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   461
        return entity, rels
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   462
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   463
    def init_entity(self, entity):
11303
891f3ecdbf7f [dataimport] rephrase minor bits of the meta generator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11142
diff changeset
   464
        # if cwuri is specified, this is an extid. It's not if it's generated in the above loop
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   465
        extid = entity.cw_edited.get('cwuri')
11303
891f3ecdbf7f [dataimport] rephrase minor bits of the meta generator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11142
diff changeset
   466
        if isinstance(extid, text_type):
891f3ecdbf7f [dataimport] rephrase minor bits of the meta generator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11142
diff changeset
   467
            extid = extid.encode('utf-8')
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   468
        for attr in self.entity_attrs:
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   469
            if attr in entity.cw_edited:
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   470
                # already set, skip this attribute
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   471
                continue
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   472
            genfunc = self.generate(attr)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   473
            if genfunc:
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   474
                entity.cw_edited.edited_attribute(attr, genfunc(entity))
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   475
        return self.source, extid
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   476
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   477
    def generate(self, rtype):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   478
        return getattr(self, 'gen_%s' % rtype, None)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   479
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   480
    def gen_cwuri(self, entity):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   481
        assert self.baseurl, 'baseurl is None while generating cwuri'
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   482
        return u'%s%s' % (self.baseurl, entity.eid)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   483
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   484
    def gen_creation_date(self, entity):
11142
45f738a634e5 [dataimport] rename .time to .now attribute of meta generator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11140
diff changeset
   485
        return self._now
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   486
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   487
    def gen_modification_date(self, entity):
11142
45f738a634e5 [dataimport] rename .time to .now attribute of meta generator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11140
diff changeset
   488
        return self._now
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   489
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   490
    def gen_created_by(self, entity):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   491
        return self._cnx.user.eid
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   492
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   493
    def gen_owned_by(self, entity):
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   494
        return self._cnx.user.eid