cubicweb/dataimport/test/test_stores.py
author Denis Laxalde <denis.laxalde@logilab.fr>
Wed, 05 Apr 2017 14:59:09 +0200
branch3.25
changeset 12146 d540defa0591
parent 11763 39df042f4ab4
permissions -rw-r--r--
[server] Add source_by_eid and source_by_uri methods to repository Most of the times we only need to retrieve one source (either by uri or eid) and querying sources_by_eid and sources_by_uri properties on repository just for one item is costly. So these methods query what's needed. We issue a ValueError (instead of KeyError for sources_by_{eid,uri} dict) in case the key is not found.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
11306
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
     1
# copyright 2003-2016 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
     3
#
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
     4
# This file is part of CubicWeb.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
     5
#
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
     9
# any later version.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
    10
#
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
    14
# details.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
    15
#
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
    18
"""unittest for cubicweb.dataimport.stores"""
10294
277074659cad [dataimport] don't generate metadata which are explicitly specified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10286
diff changeset
    19
9903
d1fdbdbab194 [dataimport] _create_copyfrom_buffer: add the tests
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 9695
diff changeset
    20
import datetime as DT
10294
277074659cad [dataimport] don't generate metadata which are explicitly specified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10286
diff changeset
    21
11308
df75fe529ba8 [dataimport] Prefer now(pytz.utc) to utcnow
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11306
diff changeset
    22
import pytz
df75fe529ba8 [dataimport] Prefer now(pytz.utc) to utcnow
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11306
diff changeset
    23
10513
7bec01a59f92 [dataimport] dispatch and deprecate old code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10459
diff changeset
    24
from cubicweb.dataimport import stores
9909
e2d42ad10f98 [test] Add test for dataimport's RQLObjectStore
Julien Cristau <julien.cristau@logilab.fr>
parents: 9903
diff changeset
    25
from cubicweb.devtools.testlib import CubicWebTC
e2d42ad10f98 [test] Add test for dataimport's RQLObjectStore
Julien Cristau <julien.cristau@logilab.fr>
parents: 9903
diff changeset
    26
e2d42ad10f98 [test] Add test for dataimport's RQLObjectStore
Julien Cristau <julien.cristau@logilab.fr>
parents: 9903
diff changeset
    27
e2d42ad10f98 [test] Add test for dataimport's RQLObjectStore
Julien Cristau <julien.cristau@logilab.fr>
parents: 9903
diff changeset
    28
class RQLObjectStoreTC(CubicWebTC):
11306
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    29
    store_impl = stores.RQLObjectStore
11327
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    30
    insert_group_attrs = dict(name=u'grp')
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    31
    insert_user_attrs = dict(login=u'lgn', upassword=u'pwd')
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    32
    source_name = 'system'
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    33
    user_extid = None
9909
e2d42ad10f98 [test] Add test for dataimport's RQLObjectStore
Julien Cristau <julien.cristau@logilab.fr>
parents: 9903
diff changeset
    34
11306
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    35
    def test_base(self):
9909
e2d42ad10f98 [test] Add test for dataimport's RQLObjectStore
Julien Cristau <julien.cristau@logilab.fr>
parents: 9903
diff changeset
    36
        with self.admin_access.repo_cnx() as cnx:
11306
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    37
            store = self.store_impl(cnx)
10457
1f5026e7d848 [dataimport] Move stores to new API.
Yann Voté <yann.vote@logilab.fr>
parents: 10350
diff changeset
    38
            # Check data insertion
11327
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    39
            group_eid = store.prepare_insert_entity('CWGroup', **self.insert_group_attrs)
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    40
            user_eid = store.prepare_insert_entity('CWUser', **self.insert_user_attrs)
10459
5ccc3bd8927e [test] Use store.prepare_insert_relation instead of deprecated relate method
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 10457
diff changeset
    41
            store.prepare_insert_relation(user_eid, 'in_group', group_eid)
11328
9f2d7da47526 [dataimport] test and fix external source support for the massive store
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11327
diff changeset
    42
            store.flush()
11306
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    43
            store.commit()
11328
9f2d7da47526 [dataimport] test and fix external source support for the massive store
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11327
diff changeset
    44
            store.finish()
11306
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    45
            user = cnx.execute('CWUser X WHERE X login "lgn"').one()
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    46
            self.assertEqual(user_eid, user.eid)
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    47
            self.assertTrue(user.creation_date)
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    48
            self.assertTrue(user.modification_date)
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    49
            self.assertTrue(user.cwuri)
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    50
            self.assertEqual(user.created_by[0].eid, cnx.user.eid)
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    51
            self.assertEqual(user.owned_by[0].eid, cnx.user.eid)
11327
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    52
            self.assertEqual(user.cw_source[0].name, self.source_name)
9909
e2d42ad10f98 [test] Add test for dataimport's RQLObjectStore
Julien Cristau <julien.cristau@logilab.fr>
parents: 9903
diff changeset
    53
            groups = cnx.execute('CWGroup X WHERE U in_group X, U login "lgn"')
e2d42ad10f98 [test] Add test for dataimport's RQLObjectStore
Julien Cristau <julien.cristau@logilab.fr>
parents: 9903
diff changeset
    54
            self.assertEqual(group_eid, groups.one().eid)
10457
1f5026e7d848 [dataimport] Move stores to new API.
Yann Voté <yann.vote@logilab.fr>
parents: 10350
diff changeset
    55
            # Check data update
1f5026e7d848 [dataimport] Move stores to new API.
Yann Voté <yann.vote@logilab.fr>
parents: 10350
diff changeset
    56
            store.prepare_update_entity('CWGroup', group_eid, name=u'new_grp')
11306
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    57
            store.commit()
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    58
            self.assertFalse(cnx.execute('CWGroup X WHERE X name "grp"'))
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    59
            self.assertTrue(cnx.execute('CWGroup X WHERE X name "new_grp"'))
10457
1f5026e7d848 [dataimport] Move stores to new API.
Yann Voté <yann.vote@logilab.fr>
parents: 10350
diff changeset
    60
            # Check data update with wrong type
1f5026e7d848 [dataimport] Move stores to new API.
Yann Voté <yann.vote@logilab.fr>
parents: 10350
diff changeset
    61
            with self.assertRaises(AssertionError):
1f5026e7d848 [dataimport] Move stores to new API.
Yann Voté <yann.vote@logilab.fr>
parents: 10350
diff changeset
    62
                store.prepare_update_entity('CWUser', group_eid, name=u'new_user')
11306
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    63
            store.commit()
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    64
            self.assertFalse(cnx.execute('CWGroup X WHERE X name "new_user"'))
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    65
            self.assertTrue(cnx.execute('CWGroup X WHERE X name "new_grp"'))
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    66
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    67
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    68
class NoHookRQLObjectStoreTC(RQLObjectStoreTC):
e759631889be [dataimport] make rql store test easier to read and test a few more thing, then reuse it to test nohook start which had no tests at all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11165
diff changeset
    69
    store_impl = stores.NoHookRQLObjectStore
9695
aa982b7c3f2a [dataimport] Prevent ucsvreader from skipping the first line when ignore_errors is True (closes #3705791)
Rémi Cardona <remi.cardona@logilab.fr>
parents: 9181
diff changeset
    70
10294
277074659cad [dataimport] don't generate metadata which are explicitly specified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10286
diff changeset
    71
11327
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    72
class NoHookRQLObjectStoreWithCustomMDGenStoreTC(RQLObjectStoreTC):
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    73
    insert_group_attrs = RQLObjectStoreTC.insert_group_attrs.copy()
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    74
    insert_group_attrs['cwuri'] = u'http://somewhere.com/group/1'
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    75
    insert_user_attrs = RQLObjectStoreTC.insert_user_attrs.copy()
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    76
    insert_user_attrs['cwuri'] = u'http://somewhere.com/user/1'
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    77
    source_name = 'test'
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    78
    user_extid = b'http://somewhere.com/user/1'
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    79
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    80
    def store_impl(self, cnx):
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    81
        source = cnx.create_entity('CWSource', type=u'datafeed', name=u'test', url=u'test')
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    82
        cnx.commit()
12146
d540defa0591 [server] Add source_by_eid and source_by_uri methods to repository
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11763
diff changeset
    83
        metagen = stores.MetadataGenerator(cnx, source=cnx.repo.source_by_eid(source.eid))
11327
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    84
        return stores.NoHookRQLObjectStore(cnx, metagen)
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    85
901243e41152 [dataimport] add new test for nohook store importing in a custom source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11309
diff changeset
    86
10286
0f8c3ac88f1e [dataimport] don't insert created_by / owned_by relations when user is the internal manager (eid = -1).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9909
diff changeset
    87
class MetaGeneratorTC(CubicWebTC):
11309
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
    88
    metagenerator_impl = stores.MetaGenerator
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
    89
    _etype_rels = staticmethod(lambda x: x.etype_rels)
10294
277074659cad [dataimport] don't generate metadata which are explicitly specified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10286
diff changeset
    90
10286
0f8c3ac88f1e [dataimport] don't insert created_by / owned_by relations when user is the internal manager (eid = -1).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9909
diff changeset
    91
    def test_dont_generate_relation_to_internal_manager(self):
0f8c3ac88f1e [dataimport] don't insert created_by / owned_by relations when user is the internal manager (eid = -1).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9909
diff changeset
    92
        with self.admin_access.repo_cnx() as cnx:
11309
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
    93
            metagen = self.metagenerator_impl(cnx)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
    94
            self.assertIn('created_by', self._etype_rels(metagen))
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
    95
            self.assertIn('owned_by', self._etype_rels(metagen))
10286
0f8c3ac88f1e [dataimport] don't insert created_by / owned_by relations when user is the internal manager (eid = -1).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9909
diff changeset
    96
        with self.repo.internal_cnx() as cnx:
11309
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
    97
            metagen = self.metagenerator_impl(cnx)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
    98
            self.assertNotIn('created_by', self._etype_rels(metagen))
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
    99
            self.assertNotIn('owned_by', self._etype_rels(metagen))
10286
0f8c3ac88f1e [dataimport] don't insert created_by / owned_by relations when user is the internal manager (eid = -1).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9909
diff changeset
   100
10294
277074659cad [dataimport] don't generate metadata which are explicitly specified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10286
diff changeset
   101
    def test_dont_generate_specified_values(self):
277074659cad [dataimport] don't generate metadata which are explicitly specified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10286
diff changeset
   102
        with self.admin_access.repo_cnx() as cnx:
11309
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   103
            metagen = self.metagenerator_impl(cnx)
10294
277074659cad [dataimport] don't generate metadata which are explicitly specified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10286
diff changeset
   104
            # hijack gen_modification_date to ensure we don't go through it
277074659cad [dataimport] don't generate metadata which are explicitly specified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10286
diff changeset
   105
            metagen.gen_modification_date = None
11308
df75fe529ba8 [dataimport] Prefer now(pytz.utc) to utcnow
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11306
diff changeset
   106
            md = DT.datetime.now(pytz.utc) - DT.timedelta(days=1)
10294
277074659cad [dataimport] don't generate metadata which are explicitly specified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10286
diff changeset
   107
            entity, rels = metagen.base_etype_dicts('CWUser')
277074659cad [dataimport] don't generate metadata which are explicitly specified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10286
diff changeset
   108
            entity.cw_edited.update(dict(modification_date=md))
11429
6a9a9ea1e9b9 Remove usage of cnx.ensure_cnx_set context manager
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11333
diff changeset
   109
            metagen.init_entity(entity)
10294
277074659cad [dataimport] don't generate metadata which are explicitly specified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10286
diff changeset
   110
            self.assertEqual(entity.cw_edited['modification_date'], md)
277074659cad [dataimport] don't generate metadata which are explicitly specified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10286
diff changeset
   111
277074659cad [dataimport] don't generate metadata which are explicitly specified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10286
diff changeset
   112
11309
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   113
class MetadataGeneratorWrapperTC(MetaGeneratorTC):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   114
    @staticmethod
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   115
    def metagenerator_impl(cnx):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   116
        return stores._MetaGeneratorBWCompatWrapper(stores.MetadataGenerator(cnx))
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   117
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   118
    _etype_rels = staticmethod(lambda x: x._mdgen._etype_rels)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   119
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   120
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   121
class MetadataGeneratorTC(CubicWebTC):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   122
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   123
    def test_dont_generate_relation_to_internal_manager(self):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   124
        with self.admin_access.repo_cnx() as cnx:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   125
            metagen = stores.MetadataGenerator(cnx)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   126
            self.assertIn('created_by', metagen.etype_rels('CWUser'))
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   127
            self.assertIn('owned_by', metagen.etype_rels('CWUser'))
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   128
        with self.repo.internal_cnx() as cnx:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   129
            metagen = stores.MetadataGenerator(cnx)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   130
            self.assertNotIn('created_by', metagen.etype_rels('CWUser'))
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   131
            self.assertNotIn('owned_by', metagen.etype_rels('CWUser'))
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   132
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   133
    def test_dont_generate_specified_values(self):
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   134
        with self.admin_access.repo_cnx() as cnx:
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   135
            metagen = stores.MetadataGenerator(cnx)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   136
            # hijack gen_modification_date to ensure we don't go through it
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   137
            metagen.gen_modification_date = None
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   138
            md = DT.datetime.now(pytz.utc) - DT.timedelta(days=1)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   139
            attrs = metagen.base_etype_attrs('CWUser')
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   140
            attrs.update(dict(modification_date=md))
11429
6a9a9ea1e9b9 Remove usage of cnx.ensure_cnx_set context manager
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11333
diff changeset
   141
            metagen.init_entity_attrs('CWUser', 1, attrs)
11309
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   142
            self.assertEqual(attrs['modification_date'], md)
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   143
31bf3254be69 [dataimport] introduce a new MetadataGenerator to replace former MetaGenerator
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11308
diff changeset
   144
9181
2eac0aa1d3f6 [dataimport] ucsvreader should skip empty lines unless specified otherwise. Closes #3035944
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   145
if __name__ == '__main__':
11333
094b29cf2dc9 [test] Prefer unittest.main
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11328
diff changeset
   146
    import unittest
094b29cf2dc9 [test] Prefer unittest.main
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11328
diff changeset
   147
    unittest.main()