server/test/unittest_datafeed.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Thu, 20 Mar 2014 08:49:10 +0100
changeset 9822 4a118bfd6ab4
parent 9798 1c105cf42904
child 9824 30183ecf5c61
permissions -rw-r--r--
[datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source This is necessary because in some case one will want to write parser which create entities not one-to-one related to entities from the distant source (for instance see the `vcrs` cube which create entities holding statistics for code review, while not importing review entities themselves). Ideally we should provide a way to the parser to indicate the source that cwuri should not be used, but that's a start... Closes #3670210
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
9798
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
     1
# copyright 2011-2014 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     3
#
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     4
# This file is part of CubicWeb.
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     5
#
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     9
# any later version.
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    10
#
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    14
# details.
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    15
#
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    18
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    19
from datetime import timedelta
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    20
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    21
from cubicweb.devtools.testlib import CubicWebTC
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    22
from cubicweb.server.sources import datafeed
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    23
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    24
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    25
class DataFeedTC(CubicWebTC):
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    26
    def setup_database(self):
9798
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    27
        with self.admin_access.repo_cnx() as cnx:
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    28
            cnx.create_entity('CWSource', name=u'myfeed', type=u'datafeed',
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    29
                              parser=u'testparser', url=u'ignored',
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    30
                              config=u'synchronization-interval=1min')
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    31
            cnx.commit()
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    32
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    33
    def test(self):
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    34
        self.assertIn('myfeed', self.repo.sources_by_uri)
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    35
        dfsource = self.repo.sources_by_uri['myfeed']
9822
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
    36
        self.assertNotIn('use_cwuri_as_url', dfsource.__dict__)
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
    37
        self.assertEqual({'type': u'datafeed', 'uri': u'myfeed', 'use-cwuri-as-url': True},
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
    38
                         dfsource.public_config)
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
    39
        self.assertEqual(dfsource.use_cwuri_as_url, True)
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    40
        self.assertEqual(dfsource.latest_retrieval, None)
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    41
        self.assertEqual(dfsource.synchro_interval, timedelta(seconds=60))
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    42
        self.assertFalse(dfsource.fresh())
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    43
7978
b6150210f692 [test] fix datafeed test, we have to use an internal session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7735
diff changeset
    44
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    45
        class AParser(datafeed.DataFeedParser):
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    46
            __regid__ = 'testparser'
7386
206890413858 [repo test] datafeed parser's process method should now take raise_on_error argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
    47
            def process(self, url, raise_on_error=False):
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    48
                entity = self.extid2entity('http://www.cubicweb.org/', 'Card',
7978
b6150210f692 [test] fix datafeed test, we have to use an internal session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7735
diff changeset
    49
                                           item={'title': u'cubicweb.org',
b6150210f692 [test] fix datafeed test, we have to use an internal session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7735
diff changeset
    50
                                                 'content': u'the cw web site'})
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    51
                if not self.created_during_pull(entity):
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    52
                    self.notify_updated(entity)
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    53
            def before_entity_copy(self, entity, sourceparams):
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    54
                entity.cw_edited.update(sourceparams['item'])
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    55
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    56
        with self.temporary_appobjects(AParser):
9798
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    57
            with self.repo.internal_cnx() as cnx:
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    58
                stats = dfsource.pull_data(cnx, force=True)
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    59
                cnx.commit()
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    60
                # test import stats
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    61
                self.assertEqual(sorted(stats), ['checked', 'created', 'updated'])
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    62
                self.assertEqual(len(stats['created']), 1)
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    63
                entity = cnx.execute('Card X').get_entity(0, 0)
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    64
                self.assertIn(entity.eid, stats['created'])
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    65
                self.assertEqual(stats['updated'], set())
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    66
                # test imported entities
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    67
                self.assertEqual(entity.title, 'cubicweb.org')
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    68
                self.assertEqual(entity.content, 'the cw web site')
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    69
                self.assertEqual(entity.cwuri, 'http://www.cubicweb.org/')
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    70
                self.assertEqual(entity.cw_source[0].name, 'myfeed')
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    71
                self.assertEqual(entity.cw_metainformation(),
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    72
                                 {'type': 'Card',
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    73
                                  'source': {'uri': 'myfeed', 'type': 'datafeed', 'use-cwuri-as-url': True},
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    74
                                  'extid': 'http://www.cubicweb.org/'}
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    75
                                 )
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    76
                self.assertEqual(entity.absolute_url(), 'http://www.cubicweb.org/')
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    77
                # test repo cache keys
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    78
                self.assertEqual(self.repo._type_source_cache[entity.eid],
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    79
                                 ('Card', 'http://www.cubicweb.org/', 'myfeed'))
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    80
                self.assertEqual(self.repo._extid_cache['http://www.cubicweb.org/'],
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    81
                                 entity.eid)
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    82
                # test repull
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    83
                stats = dfsource.pull_data(cnx, force=True)
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    84
                self.assertEqual(stats['created'], set())
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    85
                self.assertEqual(stats['updated'], set((entity.eid,)))
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    86
                # test repull with caches reseted
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    87
                self.repo._type_source_cache.clear()
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    88
                self.repo._extid_cache.clear()
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    89
                stats = dfsource.pull_data(cnx, force=True)
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    90
                self.assertEqual(stats['created'], set())
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    91
                self.assertEqual(stats['updated'], set((entity.eid,)))
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    92
                self.assertEqual(self.repo._type_source_cache[entity.eid],
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    93
                                 ('Card', 'http://www.cubicweb.org/', 'myfeed'))
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    94
                self.assertEqual(self.repo._extid_cache['http://www.cubicweb.org/'],
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
    95
                                 entity.eid)
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    96
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    97
        self.assertEqual(dfsource.source_cwuris(self.session),
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    98
                         {'http://www.cubicweb.org/': (entity.eid, 'Card')}
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    99
                         )
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   100
        self.assertTrue(dfsource.latest_retrieval)
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   101
        self.assertTrue(dfsource.fresh())
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   102
7735
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7717
diff changeset
   103
        # test_rename_source
9798
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   104
        with self.admin_access.repo_cnx() as cnx:
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   105
            cnx.execute('SET S name "myrenamedfeed" WHERE S is CWSource, S name "myfeed"')
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   106
            cnx.commit()
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   107
            entity = cnx.execute('Card X').get_entity(0, 0)
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   108
            self.assertEqual(entity.cwuri, 'http://www.cubicweb.org/')
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   109
            self.assertEqual(entity.cw_source[0].name, 'myrenamedfeed')
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   110
            self.assertEqual(entity.cw_metainformation(),
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   111
                             {'type': 'Card',
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   112
                              'source': {'uri': 'myrenamedfeed', 'type': 'datafeed', 'use-cwuri-as-url': True},
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   113
                              'extid': 'http://www.cubicweb.org/'}
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   114
                             )
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   115
            self.assertEqual(self.repo._type_source_cache[entity.eid],
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   116
                             ('Card', 'http://www.cubicweb.org/', 'myrenamedfeed'))
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   117
            self.assertEqual(self.repo._extid_cache['http://www.cubicweb.org/'],
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   118
                             entity.eid)
7735
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7717
diff changeset
   119
9798
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   120
            # test_delete_source
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   121
            cnx.execute('DELETE CWSource S WHERE S name "myrenamedfeed"')
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   122
            cnx.commit()
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   123
            self.assertFalse(cnx.execute('Card X WHERE X title "cubicweb.org"'))
1c105cf42904 [tests/datafeed] use the new connection api
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9469
diff changeset
   124
            self.assertFalse(cnx.execute('Any X WHERE X has_text "cubicweb.org"'))
7650
278fe9c1f3ad [repo] closes #1821172: fix source deletion
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7631
diff changeset
   125
9822
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
   126
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
   127
class DataFeedConfigTC(CubicWebTC):
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
   128
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
   129
    def test_use_cwuri_as_url_override(self):
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
   130
        with self.admin_access.client_cnx() as cnx:
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
   131
            cnx.create_entity('CWSource', name=u'myfeed', type=u'datafeed',
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
   132
                              parser=u'testparser', url=u'ignored',
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
   133
                              config=u'use-cwuri-as-url=no')
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
   134
            cnx.commit()
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
   135
        dfsource = self.repo.sources_by_uri['myfeed']
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
   136
        self.assertEqual(dfsource.use_cwuri_as_url, False)
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
   137
        self.assertEqual({'type': u'datafeed', 'uri': u'myfeed', 'use-cwuri-as-url': False},
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
   138
                         dfsource.public_config)
4a118bfd6ab4 [datafeed] Allow to override use_cwuri_as_url in configuration of a datafeed source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9798
diff changeset
   139
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   140
if __name__ == '__main__':
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   141
    from logilab.common.testlib import unittest_main
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   142
    unittest_main()