author | Sylvain Thénault <sylvain.thenault@logilab.fr> |
Wed, 19 Oct 2016 19:29:24 +0200 | |
branch | 3.23 |
changeset 11721 | 8b287ad7b95b |
parent 11309 | 31bf3254be69 |
child 11749 | ae9789d77ea0 |
permissions | -rw-r--r-- |
11140
fabcd1c6dcd1
[dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11057
diff
changeset
|
1 |
# copyright 2003-2016 LOGILAB S.A. (Paris, FRANCE), all rights reserved. |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
2 |
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
3 |
# |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
4 |
# This file is part of CubicWeb. |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
5 |
# |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
6 |
# CubicWeb is free software: you can redistribute it and/or modify it under the |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
7 |
# terms of the GNU Lesser General Public License as published by the Free |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
8 |
# Software Foundation, either version 2.1 of the License, or (at your option) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
9 |
# any later version. |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
10 |
# |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
11 |
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
12 |
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
13 |
# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
14 |
# details. |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
15 |
# |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
16 |
# You should have received a copy of the GNU Lesser General Public License along |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
17 |
# with CubicWeb. If not, see <http://www.gnu.org/licenses/>. |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
18 |
""" |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
19 |
Stores are responsible to insert properly formatted entities and relations into the database. They |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
20 |
have the following API:: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
21 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
22 |
>>> user_eid = store.prepare_insert_entity('CWUser', login=u'johndoe') |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
23 |
>>> group_eid = store.prepare_insert_entity('CWUser', name=u'unknown') |
10794
0b4d6beec132
[dataimport] Update doc to reflect new API
Rémi Cardona <remi.cardona@logilab.fr>
parents:
10662
diff
changeset
|
24 |
>>> store.prepare_insert_relation(user_eid, 'in_group', group_eid) |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
25 |
>>> store.flush() |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
26 |
>>> store.commit() |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
27 |
>>> store.finish() |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
28 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
29 |
Some store **requires a flush** to copy data in the database, so if you want to have store |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
30 |
independant code you should explicitly call it. (There may be multiple flushes during the |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
31 |
process, or only one at the end if there is no memory issue). This is different from the |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
32 |
commit which validates the database transaction. At last, the `finish()` method should be called in |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
33 |
case the store requires additional work once everything is done. |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
34 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
35 |
* ``prepare_insert_entity(<entity type>, **kwargs) -> eid``: given an entity |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
36 |
type, attributes and inlined relations, return the eid of the entity to be |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
37 |
inserted, *with no guarantee that anything has been inserted in database*, |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
38 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
39 |
* ``prepare_update_entity(<entity type>, eid, **kwargs) -> None``: given an |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
40 |
entity type and eid, promise for update given attributes and inlined |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
41 |
relations *with no guarantee that anything has been inserted in database*, |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
42 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
43 |
* ``prepare_insert_relation(eid_from, rtype, eid_to) -> None``: indicate that a |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
44 |
relation ``rtype`` should be added between entities with eids ``eid_from`` |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
45 |
and ``eid_to``. Similar to ``prepare_insert_entity()``, *there is no |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
46 |
guarantee that the relation will be inserted in database*, |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
47 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
48 |
* ``flush() -> None``: flush any temporary data to database. May be called |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
49 |
several times during an import, |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
50 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
51 |
* ``commit() -> None``: commit the database transaction, |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
52 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
53 |
* ``finish() -> None``: additional stuff to do after import is terminated. |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
54 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
55 |
.. autoclass:: cubicweb.dataimport.stores.RQLObjectStore |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
56 |
.. autoclass:: cubicweb.dataimport.stores.NoHookRQLObjectStore |
11309
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
57 |
.. autoclass:: cubicweb.dataimport.stores.MetadataGenerator |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
58 |
""" |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
59 |
import inspect |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
60 |
import warnings |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
61 |
from datetime import datetime |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
62 |
from copy import copy |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
63 |
|
11309
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
64 |
from six import text_type, add_metaclass |
10809
359cbdf3a515
[dataimport] extid must be a bytes object
Julien Cristau <julien.cristau@logilab.fr>
parents:
10794
diff
changeset
|
65 |
|
11308
df75fe529ba8
[dataimport] Prefer now(pytz.utc) to utcnow
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11307
diff
changeset
|
66 |
import pytz |
df75fe529ba8
[dataimport] Prefer now(pytz.utc) to utcnow
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11307
diff
changeset
|
67 |
|
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
68 |
from logilab.common.decorators import cached |
11309
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
69 |
from logilab.common.deprecation import deprecated, class_deprecated |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
70 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
71 |
from cubicweb.schema import META_RTYPES, VIRTUAL_RTYPES |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
72 |
from cubicweb.server.edition import EditedEntity |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
73 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
74 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
75 |
class RQLObjectStore(object): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
76 |
"""Store that works by making RQL queries, hence with all the cubicweb's machinery activated. |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
77 |
""" |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
78 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
79 |
def __init__(self, cnx, commit=None): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
80 |
if commit is not None: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
81 |
warnings.warn('[3.19] commit argument should not be specified ' |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
82 |
'as the cnx object already provides it.', |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
83 |
DeprecationWarning, stacklevel=2) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
84 |
self._cnx = cnx |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
85 |
self._commit = commit or cnx.commit |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
86 |
# XXX 3.21 deprecated attributes |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
87 |
self.eids = {} |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
88 |
self.types = {} |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
89 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
90 |
def rql(self, *args): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
91 |
"""Execute a RQL query. This is NOT part of the store API.""" |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
92 |
return self._cnx.execute(*args) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
93 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
94 |
def prepare_insert_entity(self, *args, **kwargs): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
95 |
"""Given an entity type, attributes and inlined relations, returns the inserted entity's |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
96 |
eid. |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
97 |
""" |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
98 |
entity = self._cnx.create_entity(*args, **kwargs) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
99 |
self.eids[entity.eid] = entity |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
100 |
self.types.setdefault(args[0], []).append(entity.eid) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
101 |
return entity.eid |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
102 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
103 |
def prepare_update_entity(self, etype, eid, **kwargs): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
104 |
"""Given an entity type and eid, updates the corresponding entity with specified attributes |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
105 |
and inlined relations. |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
106 |
""" |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
107 |
entity = self._cnx.entity_from_eid(eid) |
10939
b30c2f49da57
[dataimport] Format strings with % instead of .format()
Denis Laxalde <denis.laxalde@logilab.fr>
parents:
10907
diff
changeset
|
108 |
assert entity.cw_etype == etype, 'Trying to update with wrong type %s' % etype |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
109 |
# XXX some inlined relations may already exists |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
110 |
entity.cw_set(**kwargs) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
111 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
112 |
def prepare_insert_relation(self, eid_from, rtype, eid_to, **kwargs): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
113 |
"""Insert into the database a relation ``rtype`` between entities with eids ``eid_from`` |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
114 |
and ``eid_to``. |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
115 |
""" |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
116 |
self.rql('SET X %s Y WHERE X eid %%(x)s, Y eid %%(y)s' % rtype, |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
117 |
{'x': int(eid_from), 'y': int(eid_to)}) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
118 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
119 |
def flush(self): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
120 |
"""Nothing to flush for this store.""" |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
121 |
pass |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
122 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
123 |
def commit(self): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
124 |
"""Commit the database transaction.""" |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
125 |
return self._commit() |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
126 |
|
10862
7357b1485795
[dataimport] .finish method was missing from (NoHook)RQLObjectStore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
10809
diff
changeset
|
127 |
def finish(self): |
7357b1485795
[dataimport] .finish method was missing from (NoHook)RQLObjectStore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
10809
diff
changeset
|
128 |
"""Nothing to do once import is terminated for this store.""" |
7357b1485795
[dataimport] .finish method was missing from (NoHook)RQLObjectStore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
10809
diff
changeset
|
129 |
pass |
7357b1485795
[dataimport] .finish method was missing from (NoHook)RQLObjectStore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
10809
diff
changeset
|
130 |
|
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
131 |
@property |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
132 |
def session(self): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
133 |
warnings.warn('[3.19] deprecated property.', DeprecationWarning, stacklevel=2) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
134 |
return self._cnx.repo._get_session(self._cnx.sessionid) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
135 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
136 |
@deprecated("[3.19] use cnx.find(*args, **kwargs).entities() instead") |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
137 |
def find_entities(self, *args, **kwargs): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
138 |
return self._cnx.find(*args, **kwargs).entities() |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
139 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
140 |
@deprecated("[3.19] use cnx.find(*args, **kwargs).one() instead") |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
141 |
def find_one_entity(self, *args, **kwargs): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
142 |
return self._cnx.find(*args, **kwargs).one() |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
143 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
144 |
@deprecated('[3.21] use prepare_insert_entity instead') |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
145 |
def create_entity(self, *args, **kwargs): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
146 |
eid = self.prepare_insert_entity(*args, **kwargs) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
147 |
return self._cnx.entity_from_eid(eid) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
148 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
149 |
@deprecated('[3.21] use prepare_insert_relation instead') |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
150 |
def relate(self, eid_from, rtype, eid_to, **kwargs): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
151 |
self.prepare_insert_relation(eid_from, rtype, eid_to, **kwargs) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
152 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
153 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
154 |
class NoHookRQLObjectStore(RQLObjectStore): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
155 |
"""Store that works by accessing low-level CubicWeb's source API, with all hooks deactivated. It |
11307
74f5814ecdf0
[dataimport] rename nohook store .source attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11304
diff
changeset
|
156 |
may be given a metadata generator object to handle metadata which are usually handled by hooks. |
74f5814ecdf0
[dataimport] rename nohook store .source attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11304
diff
changeset
|
157 |
|
74f5814ecdf0
[dataimport] rename nohook store .source attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11304
diff
changeset
|
158 |
Arguments: |
74f5814ecdf0
[dataimport] rename nohook store .source attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11304
diff
changeset
|
159 |
- `cnx`, a connection to the repository |
11309
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
160 |
- `metagen`, optional :class:`MetadataGenerator` instance |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
161 |
""" |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
162 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
163 |
def __init__(self, cnx, metagen=None): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
164 |
super(NoHookRQLObjectStore, self).__init__(cnx) |
11309
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
165 |
if metagen is None: |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
166 |
metagen = MetadataGenerator(cnx) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
167 |
if isinstance(metagen, MetadataGenerator): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
168 |
metagen = _MetaGeneratorBWCompatWrapper(metagen) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
169 |
self.metagen = metagen |
11307
74f5814ecdf0
[dataimport] rename nohook store .source attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11304
diff
changeset
|
170 |
self._system_source = cnx.repo.system_source |
11140
fabcd1c6dcd1
[dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11057
diff
changeset
|
171 |
self._rschema = cnx.repo.schema.rschema |
11309
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
172 |
self._create_eid = self._system_source.create_eid |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
173 |
self._add_relation = self._system_source.add_relation |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
174 |
self._nb_inserted_entities = 0 |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
175 |
self._nb_inserted_types = 0 |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
176 |
self._nb_inserted_relations = 0 |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
177 |
# deactivate security |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
178 |
cnx.read_security = False |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
179 |
cnx.write_security = False |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
180 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
181 |
def prepare_insert_entity(self, etype, **kwargs): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
182 |
"""Given an entity type, attributes and inlined relations, returns the inserted entity's |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
183 |
eid. |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
184 |
""" |
10662
10942ed172de
[py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents:
10513
diff
changeset
|
185 |
for k, v in kwargs.items(): |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
186 |
kwargs[k] = getattr(v, 'eid', v) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
187 |
entity, rels = self.metagen.base_etype_dicts(etype) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
188 |
# make a copy to keep cached entity pristine |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
189 |
entity = copy(entity) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
190 |
entity.cw_edited = copy(entity.cw_edited) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
191 |
entity.cw_clear_relation_cache() |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
192 |
entity.cw_edited.update(kwargs, skipsec=False) |
11304
1dad60d54dfb
[dataimport] move eid attribution from the metagenerator back to the nohook store
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11303
diff
changeset
|
193 |
cnx = self._cnx |
1dad60d54dfb
[dataimport] move eid attribution from the metagenerator back to the nohook store
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11303
diff
changeset
|
194 |
entity.eid = self._create_eid(cnx) |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
195 |
entity_source, extid = self.metagen.init_entity(entity) |
11307
74f5814ecdf0
[dataimport] rename nohook store .source attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11304
diff
changeset
|
196 |
self._system_source.add_info(cnx, entity, entity_source, extid) |
74f5814ecdf0
[dataimport] rename nohook store .source attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11304
diff
changeset
|
197 |
self._system_source.add_entity(cnx, entity) |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
198 |
kwargs = dict() |
11140
fabcd1c6dcd1
[dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11057
diff
changeset
|
199 |
if inspect.getargspec(self._add_relation).keywords: |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
200 |
kwargs['subjtype'] = entity.cw_etype |
10662
10942ed172de
[py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents:
10513
diff
changeset
|
201 |
for rtype, targeteids in rels.items(): |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
202 |
# targeteids may be a single eid or a list of eids |
11140
fabcd1c6dcd1
[dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11057
diff
changeset
|
203 |
inlined = self._rschema(rtype).inlined |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
204 |
try: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
205 |
for targeteid in targeteids: |
11140
fabcd1c6dcd1
[dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11057
diff
changeset
|
206 |
self._add_relation(cnx, entity.eid, rtype, targeteid, |
fabcd1c6dcd1
[dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11057
diff
changeset
|
207 |
inlined, **kwargs) |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
208 |
except TypeError: |
11140
fabcd1c6dcd1
[dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11057
diff
changeset
|
209 |
self._add_relation(cnx, entity.eid, rtype, targeteids, |
fabcd1c6dcd1
[dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11057
diff
changeset
|
210 |
inlined, **kwargs) |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
211 |
self._nb_inserted_entities += 1 |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
212 |
return entity.eid |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
213 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
214 |
# XXX: prepare_update_entity is inherited from RQLObjectStore, it should be reimplemented to |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
215 |
# actually skip hooks as prepare_insert_entity |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
216 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
217 |
def prepare_insert_relation(self, eid_from, rtype, eid_to, **kwargs): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
218 |
"""Insert into the database a relation ``rtype`` between entities with eids ``eid_from`` |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
219 |
and ``eid_to``. |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
220 |
""" |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
221 |
assert not rtype.startswith('reverse_') |
11140
fabcd1c6dcd1
[dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11057
diff
changeset
|
222 |
rschema = self._rschema(rtype) |
fabcd1c6dcd1
[dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11057
diff
changeset
|
223 |
self._add_relation(self._cnx, eid_from, rtype, eid_to, rschema.inlined) |
fabcd1c6dcd1
[dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11057
diff
changeset
|
224 |
if rschema.symmetric: |
fabcd1c6dcd1
[dataimport] cleanup a bit nohook store, mostly protecting some attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11057
diff
changeset
|
225 |
self._add_relation(self._cnx, eid_to, rtype, eid_from, rschema.inlined) |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
226 |
self._nb_inserted_relations += 1 |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
227 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
228 |
@property |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
229 |
@deprecated('[3.21] deprecated') |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
230 |
def nb_inserted_entities(self): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
231 |
return self._nb_inserted_entities |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
232 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
233 |
@property |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
234 |
@deprecated('[3.21] deprecated') |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
235 |
def nb_inserted_types(self): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
236 |
return self._nb_inserted_types |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
237 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
238 |
@property |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
239 |
@deprecated('[3.21] deprecated') |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
240 |
def nb_inserted_relations(self): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
241 |
return self._nb_inserted_relations |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
242 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
243 |
|
11309
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
244 |
class MetadataGenerator(object): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
245 |
"""Class responsible for generating standard metadata for imported entities. You may want to |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
246 |
derive it to add application specific's metadata. This class (or a subclass) may either be |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
247 |
given to a nohook or massive store. |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
248 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
249 |
Parameters: |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
250 |
* `cnx`: connection to the repository |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
251 |
* `baseurl`: optional base URL to be used for `cwuri` generation - default to config['base-url'] |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
252 |
* `source`: optional source to be used as `cw_source` for imported entities |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
253 |
""" |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
254 |
META_RELATIONS = (META_RTYPES |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
255 |
- VIRTUAL_RTYPES |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
256 |
- set(('eid', 'cwuri', |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
257 |
'is', 'is_instance_of', 'cw_source'))) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
258 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
259 |
def __init__(self, cnx, baseurl=None, source=None): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
260 |
self._cnx = cnx |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
261 |
if baseurl is None: |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
262 |
config = cnx.vreg.config |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
263 |
baseurl = config['base-url'] or config.default_base_url() |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
264 |
if not baseurl[-1] == '/': |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
265 |
baseurl += '/' |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
266 |
self._baseurl = baseurl |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
267 |
if source is None: |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
268 |
source = cnx.repo.system_source |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
269 |
self.source = source |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
270 |
self._need_extid = source is not cnx.repo.system_source |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
271 |
self._now = datetime.now(pytz.utc) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
272 |
# attributes/relations shared by all entities of the same type |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
273 |
self._etype_attrs = [] |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
274 |
self._etype_rels = [] |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
275 |
# attributes/relations specific to each entity |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
276 |
self._entity_attrs = ['cwuri'] |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
277 |
rschema = cnx.vreg.schema.rschema |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
278 |
for rtype in self.META_RELATIONS: |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
279 |
# skip owned_by / created_by if user is the internal manager |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
280 |
if cnx.user.eid == -1 and rtype in ('owned_by', 'created_by'): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
281 |
continue |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
282 |
if rschema(rtype).final: |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
283 |
self._etype_attrs.append(rtype) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
284 |
else: |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
285 |
self._etype_rels.append(rtype) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
286 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
287 |
# etype is provided in the 3 methods below as proven useful to custom implementation but not |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
288 |
# used by the default implementation |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
289 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
290 |
def etype_attrs(self, etype): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
291 |
"""Return the list of attributes to be set for all entities of the given type.""" |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
292 |
return self._etype_attrs[:] |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
293 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
294 |
def etype_rels(self, etype): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
295 |
"""Return the list of relations to be set for all entities of the given type.""" |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
296 |
return self._etype_rels[:] |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
297 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
298 |
def entity_attrs(self, etype): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
299 |
"""Return the list of attributes whose value is set per instance, not per type, for the |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
300 |
given type. |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
301 |
""" |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
302 |
return self._entity_attrs[:] |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
303 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
304 |
@cached |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
305 |
def base_etype_attrs(self, etype): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
306 |
"""Return a dictionary of attributes to be set for all entities of the given type.""" |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
307 |
attrs = {} |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
308 |
for attr in self.etype_attrs(etype): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
309 |
genfunc = self._generator(attr) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
310 |
if genfunc: |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
311 |
attrs[attr] = genfunc(etype) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
312 |
return attrs |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
313 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
314 |
@cached |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
315 |
def base_etype_rels(self, etype): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
316 |
"""Return a dictionary of relations to be set for all entities of the given type.""" |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
317 |
rels = {} |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
318 |
for rel in self.etype_rels(etype): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
319 |
genfunc = self._generator(rel) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
320 |
if genfunc: |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
321 |
rels[rel] = genfunc(etype) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
322 |
return rels |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
323 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
324 |
def entity_extid(self, etype, eid, attrs): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
325 |
"""Return the extid for the entity of given type and eid, to be inserted in the 'entities' |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
326 |
system table. |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
327 |
""" |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
328 |
if self._need_extid: |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
329 |
extid = attrs.get('cwuri') |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
330 |
if extid is None: |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
331 |
raise Exception('entity from an external source but no extid specified') |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
332 |
elif isinstance(extid, text_type): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
333 |
extid = extid.encode('utf-8') |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
334 |
else: |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
335 |
extid = None |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
336 |
return extid |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
337 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
338 |
def init_entity_attrs(self, etype, eid, attrs): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
339 |
"""Insert into an entity attrs dictionary attributes whose value is set per instance, not per |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
340 |
type. |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
341 |
""" |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
342 |
for attr in self.entity_attrs(etype): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
343 |
if attr in attrs: |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
344 |
# already set, skip this attribute |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
345 |
continue |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
346 |
genfunc = self._generator(attr) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
347 |
if genfunc: |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
348 |
attrs[attr] = genfunc(etype, eid, attrs) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
349 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
350 |
def _generator(self, rtype): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
351 |
return getattr(self, 'gen_%s' % rtype, None) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
352 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
353 |
def gen_cwuri(self, etype, eid, attrs): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
354 |
assert self._baseurl, 'baseurl is None while generating cwuri' |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
355 |
return u'%s%s' % (self._baseurl, eid) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
356 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
357 |
def gen_creation_date(self, etype): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
358 |
return self._now |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
359 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
360 |
def gen_modification_date(self, etype): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
361 |
return self._now |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
362 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
363 |
def gen_created_by(self, etype): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
364 |
return self._cnx.user.eid |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
365 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
366 |
def gen_owned_by(self, etype): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
367 |
return self._cnx.user.eid |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
368 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
369 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
370 |
class _MetaGeneratorBWCompatWrapper(object): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
371 |
"""Class wrapping a MetadataGenerator to adapt it to the MetaGenerator interface. |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
372 |
""" |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
373 |
META_RELATIONS = (META_RTYPES |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
374 |
- VIRTUAL_RTYPES |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
375 |
- set(('eid', 'cwuri', |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
376 |
'is', 'is_instance_of', 'cw_source'))) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
377 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
378 |
def __init__(self, mdgenerator): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
379 |
self._mdgen = mdgenerator |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
380 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
381 |
@cached |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
382 |
def base_etype_dicts(self, etype): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
383 |
cnx = self._mdgen._cnx |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
384 |
entity = cnx.vreg['etypes'].etype_class(etype)(cnx) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
385 |
# entity are "surface" copied, avoid shared dict between copies |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
386 |
del entity.cw_extra_kwargs |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
387 |
entity.cw_edited = EditedEntity(entity) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
388 |
attrs = self._mdgen.base_etype_attrs(etype) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
389 |
entity.cw_edited.update(attrs, skipsec=False) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
390 |
rels = self._mdgen.base_etype_rels(etype) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
391 |
return entity, rels |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
392 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
393 |
def init_entity(self, entity): |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
394 |
# if cwuri is specified, this is an extid. It's not if it's generated in the above loop |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
395 |
extid = self._mdgen.entity_extid(entity.cw_etype, entity.eid, entity.cw_edited) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
396 |
attrs = dict(entity.cw_edited) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
397 |
self._mdgen.init_entity_attrs(entity.cw_etype, entity.eid, attrs) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
398 |
entity.cw_edited.update(attrs, skipsec=False) |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
399 |
return self._mdgen.source, extid |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
400 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
401 |
|
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
402 |
@add_metaclass(class_deprecated) |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
403 |
class MetaGenerator(object): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
404 |
"""Class responsible for generating standard metadata for imported entities. You may want to |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
405 |
derive it to add application specific's metadata. |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
406 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
407 |
Parameters: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
408 |
* `cnx`: connection to the repository |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
409 |
* `baseurl`: optional base URL to be used for `cwuri` generation - default to config['base-url'] |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
410 |
* `source`: optional source to be used as `cw_source` for imported entities |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
411 |
""" |
11309
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
412 |
__deprecation_warning__ = '[3.23] this class is deprecated, use MetadataGenerator instead' |
31bf3254be69
[dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11308
diff
changeset
|
413 |
|
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
414 |
META_RELATIONS = (META_RTYPES |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
415 |
- VIRTUAL_RTYPES |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
416 |
- set(('eid', 'cwuri', |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
417 |
'is', 'is_instance_of', 'cw_source'))) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
418 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
419 |
def __init__(self, cnx, baseurl=None, source=None): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
420 |
self._cnx = cnx |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
421 |
if baseurl is None: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
422 |
config = cnx.vreg.config |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
423 |
baseurl = config['base-url'] or config.default_base_url() |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
424 |
if not baseurl[-1] == '/': |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
425 |
baseurl += '/' |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
426 |
self.baseurl = baseurl |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
427 |
if source is None: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
428 |
source = cnx.repo.system_source |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
429 |
self.source = source |
11308
df75fe529ba8
[dataimport] Prefer now(pytz.utc) to utcnow
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11307
diff
changeset
|
430 |
self._now = datetime.now(pytz.utc) |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
431 |
# attributes/relations shared by all entities of the same type |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
432 |
self.etype_attrs = [] |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
433 |
self.etype_rels = [] |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
434 |
# attributes/relations specific to each entity |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
435 |
self.entity_attrs = ['cwuri'] |
11303
891f3ecdbf7f
[dataimport] rephrase minor bits of the meta generator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11142
diff
changeset
|
436 |
rschema = cnx.vreg.schema.rschema |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
437 |
for rtype in self.META_RELATIONS: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
438 |
# skip owned_by / created_by if user is the internal manager |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
439 |
if cnx.user.eid == -1 and rtype in ('owned_by', 'created_by'): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
440 |
continue |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
441 |
if rschema(rtype).final: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
442 |
self.etype_attrs.append(rtype) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
443 |
else: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
444 |
self.etype_rels.append(rtype) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
445 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
446 |
@cached |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
447 |
def base_etype_dicts(self, etype): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
448 |
entity = self._cnx.vreg['etypes'].etype_class(etype)(self._cnx) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
449 |
# entity are "surface" copied, avoid shared dict between copies |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
450 |
del entity.cw_extra_kwargs |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
451 |
entity.cw_edited = EditedEntity(entity) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
452 |
for attr in self.etype_attrs: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
453 |
genfunc = self.generate(attr) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
454 |
if genfunc: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
455 |
entity.cw_edited.edited_attribute(attr, genfunc(entity)) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
456 |
rels = {} |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
457 |
for rel in self.etype_rels: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
458 |
genfunc = self.generate(rel) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
459 |
if genfunc: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
460 |
rels[rel] = genfunc(entity) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
461 |
return entity, rels |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
462 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
463 |
def init_entity(self, entity): |
11303
891f3ecdbf7f
[dataimport] rephrase minor bits of the meta generator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11142
diff
changeset
|
464 |
# if cwuri is specified, this is an extid. It's not if it's generated in the above loop |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
465 |
extid = entity.cw_edited.get('cwuri') |
11303
891f3ecdbf7f
[dataimport] rephrase minor bits of the meta generator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11142
diff
changeset
|
466 |
if isinstance(extid, text_type): |
891f3ecdbf7f
[dataimport] rephrase minor bits of the meta generator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11142
diff
changeset
|
467 |
extid = extid.encode('utf-8') |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
468 |
for attr in self.entity_attrs: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
469 |
if attr in entity.cw_edited: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
470 |
# already set, skip this attribute |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
471 |
continue |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
472 |
genfunc = self.generate(attr) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
473 |
if genfunc: |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
474 |
entity.cw_edited.edited_attribute(attr, genfunc(entity)) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
475 |
return self.source, extid |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
476 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
477 |
def generate(self, rtype): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
478 |
return getattr(self, 'gen_%s' % rtype, None) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
479 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
480 |
def gen_cwuri(self, entity): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
481 |
assert self.baseurl, 'baseurl is None while generating cwuri' |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
482 |
return u'%s%s' % (self.baseurl, entity.eid) |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
483 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
484 |
def gen_creation_date(self, entity): |
11142
45f738a634e5
[dataimport] rename .time to .now attribute of meta generator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11140
diff
changeset
|
485 |
return self._now |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
486 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
487 |
def gen_modification_date(self, entity): |
11142
45f738a634e5
[dataimport] rename .time to .now attribute of meta generator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
11140
diff
changeset
|
488 |
return self._now |
10513
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
489 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
490 |
def gen_created_by(self, entity): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
491 |
return self._cnx.user.eid |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
492 |
|
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
493 |
def gen_owned_by(self, entity): |
7bec01a59f92
[dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff
changeset
|
494 |
return self._cnx.user.eid |