hooks/syncsources.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Tue, 30 Jun 2015 10:06:00 +0200
changeset 10914 fed8bd56f223
parent 10666 7f6b5f023884
child 10956 208c9ac8edbb
permissions -rw-r--r--
[repository] deprecate the extid2eid based multi-sources API This API is cumbersome and lead to obfuscated code because of the callback mecanism implied when some entity has to be created. Since we dropped the "true" multi-source, this mecanism is not needed anymore and one should prefer to use the dataimport API inside its parser instead. Notice the cwxml parser will trigger the deprecation warning, it should not be used anymore in favor of ad-hoc parsers.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
8556
bbe0d6985e59 [validation error] refactor validation error handling so translation is done on the web side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8190
diff changeset
     1
# copyright 2010-2012 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
     3
#
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
     4
# This file is part of CubicWeb.
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
     5
#
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
     9
# any later version.
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
    10
#
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
    14
# details.
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
    15
#
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
    18
"""hooks for repository sources synchronization"""
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
    19
10666
7f6b5f023884 [py3k] replace '_ = unicode' in global scope (closes #7589459)
Rémi Cardona <remi.cardona@logilab.fr>
parents: 9619
diff changeset
    20
from cubicweb import _
8556
bbe0d6985e59 [validation error] refactor validation error handling so translation is done on the web side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8190
diff changeset
    21
7024
f4e798cf29b4 [source hooks] filter out case where source configuration doesn't have to be reloaded
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
    22
from socket import gethostname
f4e798cf29b4 [source hooks] filter out case where source configuration doesn't have to be reloaded
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
    23
7735
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
    24
from logilab.common.decorators import clear_cache
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
    25
8556
bbe0d6985e59 [validation error] refactor validation error handling so translation is done on the web side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8190
diff changeset
    26
from cubicweb import validation_error
8190
2a3c1b787688 [vreg] move base registry implementation to logilab.common. Closes #1916014
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
    27
from cubicweb.predicates import is_instance
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
    28
from cubicweb.server import SOURCE_TYPES, hook
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    29
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    30
class SourceHook(hook.Hook):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    31
    __abstract__ = True
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    32
    category = 'cw.sources'
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    33
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    34
7399
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7049
diff changeset
    35
# repo sources synchronization #################################################
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7049
diff changeset
    36
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    37
class SourceAddedOp(hook.Operation):
7879
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7735
diff changeset
    38
    entity = None # make pylint happy
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
    39
    def postcommit_event(self):
9619
a4e97a31cc11 [hooks/syncsources] use a cnx not a session
Julien Cristau <julien.cristau@logilab.fr>
parents: 9448
diff changeset
    40
        self.cnx.repo.add_source(self.entity)
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    41
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    42
class SourceAddedHook(SourceHook):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    43
    __regid__ = 'cw.sources.added'
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    44
    __select__ = SourceHook.__select__ & is_instance('CWSource')
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    45
    events = ('after_add_entity',)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    46
    def __call__(self):
6945
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    47
        try:
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    48
            sourcecls = SOURCE_TYPES[self.entity.type]
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    49
        except KeyError:
8557
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
    50
            msg = _('Unknown source type')
8556
bbe0d6985e59 [validation error] refactor validation error handling so translation is done on the web side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8190
diff changeset
    51
            raise validation_error(self.entity, {('type', 'subject'): msg})
8557
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
    52
        # ignore creation of the system source done during database
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
    53
        # initialisation, as config for this source is in a file and handling
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
    54
        # is done separatly (no need for the operation either)
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
    55
        if self.entity.name != 'system':
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
    56
            sourcecls.check_conf_dict(self.entity.eid, self.entity.host_config,
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
    57
                                      fail_if_unknown=not self._cw.vreg.config.repairing)
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
    58
            SourceAddedOp(self._cw, entity=self.entity)
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    59
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    60
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    61
class SourceRemovedOp(hook.Operation):
7879
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7735
diff changeset
    62
    uri = None # make pylint happy
7650
278fe9c1f3ad [repo] closes #1821172: fix source deletion
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7536
diff changeset
    63
    def postcommit_event(self):
9619
a4e97a31cc11 [hooks/syncsources] use a cnx not a session
Julien Cristau <julien.cristau@logilab.fr>
parents: 9448
diff changeset
    64
        self.cnx.repo.remove_source(self.uri)
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    65
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    66
class SourceRemovedHook(SourceHook):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    67
    __regid__ = 'cw.sources.removed'
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    68
    __select__ = SourceHook.__select__ & is_instance('CWSource')
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    69
    events = ('before_delete_entity',)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    70
    def __call__(self):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    71
        if self.entity.name == 'system':
8556
bbe0d6985e59 [validation error] refactor validation error handling so translation is done on the web side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8190
diff changeset
    72
            msg = _("You cannot remove the system source")
bbe0d6985e59 [validation error] refactor validation error handling so translation is done on the web side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8190
diff changeset
    73
            raise validation_error(self.entity, {None: msg})
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    74
        SourceRemovedOp(self._cw, uri=self.entity.name)
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
    75
6945
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    76
7735
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
    77
class SourceConfigUpdatedOp(hook.DataOperationMixIn, hook.Operation):
6945
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    78
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    79
    def precommit_event(self):
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    80
        self.__processed = []
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    81
        for source in self.get_data():
9619
a4e97a31cc11 [hooks/syncsources] use a cnx not a session
Julien Cristau <julien.cristau@logilab.fr>
parents: 9448
diff changeset
    82
            if not self.cnx.deleted_in_transaction(source.eid):
7650
278fe9c1f3ad [repo] closes #1821172: fix source deletion
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7536
diff changeset
    83
                conf = source.repo_source.check_config(source)
278fe9c1f3ad [repo] closes #1821172: fix source deletion
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7536
diff changeset
    84
                self.__processed.append( (source, conf) )
6945
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    85
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    86
    def postcommit_event(self):
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    87
        for source, conf in self.__processed:
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    88
            source.repo_source.update_config(source, conf)
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    89
7735
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
    90
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
    91
class SourceRenamedOp(hook.LateOperation):
7879
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7735
diff changeset
    92
    oldname = newname = None # make pylint happy
7735
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
    93
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
    94
    def precommit_event(self):
9619
a4e97a31cc11 [hooks/syncsources] use a cnx not a session
Julien Cristau <julien.cristau@logilab.fr>
parents: 9448
diff changeset
    95
        source = self.cnx.repo.sources_by_uri[self.oldname]
9448
3e7cad3967c5 [multi-sources-removal] Drop the "true" multi-sources planner
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8557
diff changeset
    96
        sql = 'UPDATE entities SET asource=%(newname)s WHERE asource=%(oldname)s'
9619
a4e97a31cc11 [hooks/syncsources] use a cnx not a session
Julien Cristau <julien.cristau@logilab.fr>
parents: 9448
diff changeset
    97
        self.cnx.system_sql(sql, {'oldname': self.oldname,
7735
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
    98
                                      'newname': self.newname})
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
    99
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   100
    def postcommit_event(self):
9619
a4e97a31cc11 [hooks/syncsources] use a cnx not a session
Julien Cristau <julien.cristau@logilab.fr>
parents: 9448
diff changeset
   101
        repo = self.cnx.repo
7735
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   102
        # XXX race condition
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   103
        source = repo.sources_by_uri.pop(self.oldname)
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   104
        source.uri = self.newname
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   105
        source.public_config['uri'] = self.newname
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   106
        repo.sources_by_uri[self.newname] = source
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   107
        repo._type_source_cache.clear()
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   108
        clear_cache(repo, 'source_defs')
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   109
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   110
6945
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   111
class SourceUpdatedHook(SourceHook):
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   112
    __regid__ = 'cw.sources.configupdate'
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   113
    __select__ = SourceHook.__select__ & is_instance('CWSource')
7735
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   114
    events = ('before_update_entity',)
6945
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   115
    def __call__(self):
7735
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   116
        if 'name' in self.entity.cw_edited:
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   117
            oldname, newname = self.entity.cw_edited.oldnewvalue('name')
8557
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
   118
            if oldname == 'system':
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
   119
                msg = _("You cannot rename the system source")
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
   120
                raise validation_error(self.entity, {('name', 'subject'): msg})
7735
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   121
            SourceRenamedOp(self._cw, oldname=oldname, newname=newname)
8557
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
   122
        if 'config' in self.entity.cw_edited:
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
   123
            if self.entity.name == 'system' and self.entity.config:
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
   124
                msg = _("Configuration of the system source goes to "
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
   125
                        "the 'sources' file, not in the database")
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
   126
                raise validation_error(self.entity, {('config', 'subject'): msg})
8756f932ca83 [source synchronization hooks] Fix/enhance system source hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8556
diff changeset
   127
            SourceConfigUpdatedOp.get_instance(self._cw).add_data(self.entity)
7735
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   128
6945
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   129
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   130
class SourceHostConfigUpdatedHook(SourceHook):
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   131
    __regid__ = 'cw.sources.hostconfigupdate'
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   132
    __select__ = SourceHook.__select__ & is_instance('CWSourceHostConfig')
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   133
    events = ('after_add_entity', 'after_update_entity', 'before_delete_entity',)
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   134
    def __call__(self):
7049
f2ae6202754c [source host alias] match_host is the attribute, match is the method
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7024
diff changeset
   135
        if self.entity.match(gethostname()):
7024
f4e798cf29b4 [source hooks] filter out case where source configuration doesn't have to be reloaded
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
   136
            if self.event == 'after_update_entity' and \
f4e798cf29b4 [source hooks] filter out case where source configuration doesn't have to be reloaded
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
   137
                   not 'config' in self.entity.cw_edited:
f4e798cf29b4 [source hooks] filter out case where source configuration doesn't have to be reloaded
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
   138
                return
f4e798cf29b4 [source hooks] filter out case where source configuration doesn't have to be reloaded
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
   139
            try:
7735
71adfd6bab38 [source] update system table and internal structures on source renaming. Closes #1896721
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7650
diff changeset
   140
                SourceConfigUpdatedOp.get_instance(self._cw).add_data(self.entity.cwsource)
7024
f4e798cf29b4 [source hooks] filter out case where source configuration doesn't have to be reloaded
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
   141
            except IndexError:
f4e798cf29b4 [source hooks] filter out case where source configuration doesn't have to be reloaded
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
   142
                # XXX no source linked to the host config yet
f4e798cf29b4 [source hooks] filter out case where source configuration doesn't have to be reloaded
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6957
diff changeset
   143
                pass
6945
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   144
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   145
7399
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7049
diff changeset
   146
# source mapping synchronization ###############################################
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7049
diff changeset
   147
#
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7049
diff changeset
   148
# Expect cw_for_source/cw_schema are immutable relations (i.e. can't change from
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7049
diff changeset
   149
# a source or schema to another).
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   150
7879
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7735
diff changeset
   151
class SourceMappingImmutableHook(SourceHook):
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   152
    """check cw_for_source and cw_schema are immutable relations
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   153
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   154
    XXX empty delete perms would be enough?
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   155
    """
7879
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7735
diff changeset
   156
    __regid__ = 'cw.sources.mapping.immutable'
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   157
    __select__ = SourceHook.__select__ & hook.match_rtype('cw_for_source', 'cw_schema')
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   158
    events = ('before_add_relation',)
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
   159
    def __call__(self):
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   160
        if not self._cw.added_in_transaction(self.eidfrom):
8556
bbe0d6985e59 [validation error] refactor validation error handling so translation is done on the web side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8190
diff changeset
   161
            msg = _("You can't change this relation")
bbe0d6985e59 [validation error] refactor validation error handling so translation is done on the web side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8190
diff changeset
   162
            raise validation_error(self.eidfrom, {self.rtype: msg})
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   163
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   164
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   165
class SourceMappingChangedOp(hook.DataOperationMixIn, hook.Operation):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   166
    def check_or_update(self, checkonly):
9619
a4e97a31cc11 [hooks/syncsources] use a cnx not a session
Julien Cristau <julien.cristau@logilab.fr>
parents: 9448
diff changeset
   167
        cnx = self.cnx
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   168
        # take care, can't call get_data() twice
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   169
        try:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   170
            data = self.__data
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   171
        except AttributeError:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   172
            data = self.__data = self.get_data()
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   173
        for schemacfg, source in data:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   174
            if source is None:
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
   175
                source = schemacfg.cwsource.repo_source
9619
a4e97a31cc11 [hooks/syncsources] use a cnx not a session
Julien Cristau <julien.cristau@logilab.fr>
parents: 9448
diff changeset
   176
            if cnx.added_in_transaction(schemacfg.eid):
a4e97a31cc11 [hooks/syncsources] use a cnx not a session
Julien Cristau <julien.cristau@logilab.fr>
parents: 9448
diff changeset
   177
                if not cnx.deleted_in_transaction(schemacfg.eid):
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   178
                    source.add_schema_config(schemacfg, checkonly=checkonly)
9619
a4e97a31cc11 [hooks/syncsources] use a cnx not a session
Julien Cristau <julien.cristau@logilab.fr>
parents: 9448
diff changeset
   179
            elif cnx.deleted_in_transaction(schemacfg.eid):
7525
1f6839019e6a [source config] fix synchronization on configuration entity changes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7049
diff changeset
   180
                source.del_schema_config(schemacfg, checkonly=checkonly)
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   181
            else:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   182
                source.update_schema_config(schemacfg, checkonly=checkonly)
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   183
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   184
    def precommit_event(self):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   185
        self.check_or_update(True)
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   186
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   187
    def postcommit_event(self):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   188
        self.check_or_update(False)
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   189
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   190
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   191
class SourceMappingChangedHook(SourceHook):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   192
    __regid__ = 'cw.sources.schemaconfig'
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   193
    __select__ = SourceHook.__select__ & is_instance('CWSourceSchemaConfig')
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   194
    events = ('after_add_entity', 'after_update_entity')
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   195
    def __call__(self):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   196
        if self.event == 'after_add_entity' or (
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   197
            self.event == 'after_update_entity' and 'options' in self.entity.cw_edited):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   198
            SourceMappingChangedOp.get_instance(self._cw).add_data(
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   199
                (self.entity, None) )
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   200
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   201
class SourceMappingDeleteHook(SourceHook):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   202
    __regid__ = 'cw.sources.delschemaconfig'
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   203
    __select__ = SourceHook.__select__ & hook.match_rtype('cw_for_source')
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   204
    events = ('before_delete_relation',)
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   205
    def __call__(self):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   206
        SourceMappingChangedOp.get_instance(self._cw).add_data(
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   207
            (self._cw.entity_from_eid(self.eidfrom),
7525
1f6839019e6a [source config] fix synchronization on configuration entity changes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7049
diff changeset
   208
             self._cw.entity_from_eid(self.eidto).repo_source) )