web/views/cwsources.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Fri, 21 Oct 2011 09:24:48 +0200
changeset 7980 c85bea091a6c
parent 7935 28fe0b91ad8a
child 7992 4ff9f25cb06e
permissions -rw-r--r--
[test] fix failures introduced by 7951:b7c825b00f64
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
     1
# copyright 2010-2011 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     3
#
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     4
# This file is part of CubicWeb.
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     5
#
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     9
# any later version.
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    10
#
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    14
# details.
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    15
#
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
    18
"""Specific views for data sources and related entities (eg CWSource,
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
    19
CWSourceHostConfig, CWSourceSchemaConfig).
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
    20
"""
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    21
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    22
__docformat__ = "restructuredtext en"
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    23
_ = unicode
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    24
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    25
from itertools import repeat, chain
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    26
7832
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    27
from cubicweb import Unauthorized
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
    28
from cubicweb.selectors import is_instance, score_entity, match_user_groups
6962
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    29
from cubicweb.view import EntityView, StartupView
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    30
from cubicweb.schema import META_RTYPES, VIRTUAL_RTYPES, display_name
7524
c019c3426049 [ui, sources] configure source form: hide synchronizing and mapping entities, use simple text input for parser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7375
diff changeset
    31
from cubicweb.web import uicfg, formwidgets as wdgs
7832
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    32
from cubicweb.web.views import tabs, actions, ibreadcrumbs, add_etype_button
6962
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    33
7019
46413742f9b5 [source ui] uicfg tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6962
diff changeset
    34
46413742f9b5 [source ui] uicfg tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6962
diff changeset
    35
_abaa = uicfg.actionbox_appearsin_addmenu
7832
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    36
# there are explicit 'add' buttons for those
7019
46413742f9b5 [source ui] uicfg tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6962
diff changeset
    37
_abaa.tag_object_of(('CWSourceSchemaConfig', 'cw_schema', '*'), False)
46413742f9b5 [source ui] uicfg tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6962
diff changeset
    38
_abaa.tag_object_of(('CWSourceSchemaConfig', 'cw_for_source', '*'), False)
7832
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    39
_abaa.tag_object_of(('CWSourceSchemaConfig', 'cw_host_config_of', '*'), False)
7019
46413742f9b5 [source ui] uicfg tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6962
diff changeset
    40
7524
c019c3426049 [ui, sources] configure source form: hide synchronizing and mapping entities, use simple text input for parser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7375
diff changeset
    41
_afs = uicfg.autoform_section
c019c3426049 [ui, sources] configure source form: hide synchronizing and mapping entities, use simple text input for parser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7375
diff changeset
    42
_afs.tag_object_of(('*', 'cw_for_source', 'CWSource'), 'main', 'hidden')
7832
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    43
7524
c019c3426049 [ui, sources] configure source form: hide synchronizing and mapping entities, use simple text input for parser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7375
diff changeset
    44
_affk = uicfg.autoform_field_kwargs
c019c3426049 [ui, sources] configure source form: hide synchronizing and mapping entities, use simple text input for parser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7375
diff changeset
    45
_affk.tag_attribute(('CWSource', 'parser'), {'widget': wdgs.TextInput})
c019c3426049 [ui, sources] configure source form: hide synchronizing and mapping entities, use simple text input for parser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7375
diff changeset
    46
6962
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
    47
# source primary views #########################################################
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    48
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
    49
_pvs = uicfg.primaryview_section
7832
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    50
_pvs.tag_attribute(('CWSource', 'name'), 'hidden')
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
    51
_pvs.tag_object_of(('*', 'cw_for_source', 'CWSource'), 'hidden')
7832
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    52
_pvs.tag_object_of(('*', 'cw_host_config_of', 'CWSource'), 'hidden')
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    53
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    54
_pvdc = uicfg.primaryview_display_ctrl
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    55
_pvdc.tag_attribute(('CWSource', 'type'), {'vid': 'attribute'})# disable reledit
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    56
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    57
_rc = uicfg.reledit_ctrl
7847
7be550fb337b [primary view] add 'verbatim' attribute view (closes #1947475)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7832
diff changeset
    58
_rc.tag_attribute(('CWSource', 'config'), {'rvid': 'verbatimattr'})
7be550fb337b [primary view] add 'verbatim' attribute view (closes #1947475)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7832
diff changeset
    59
_rc.tag_attribute(('CWSourceHostConfig', 'config'), {'rvid': 'verbatimattr'})
7be550fb337b [primary view] add 'verbatim' attribute view (closes #1947475)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7832
diff changeset
    60
_rc.tag_attribute(('CWSourceSchemaConfig', 'options'), {'rvid': 'verbatimattr'})
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
    61
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    62
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    63
class CWSourcePrimaryView(tabs.TabbedPrimaryView):
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    64
    __select__ = is_instance('CWSource')
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    65
    tabs = [_('cwsource-main'), _('cwsource-mapping')]
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    66
    default_tab = 'cwsource-main'
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    67
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    68
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    69
class CWSourceMainTab(tabs.PrimaryTab):
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    70
    __regid__ = 'cwsource-main'
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    71
    __select__ = tabs.PrimaryTab.__select__ & is_instance('CWSource')
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    72
7832
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    73
    def render_entity_attributes(self, entity):
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    74
        super(CWSourceMainTab, self).render_entity_attributes(entity)
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    75
        self.w(add_etype_button(self._cw, 'CWSourceHostConfig',
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    76
                                __linkto='cw_host_config_of:%s:subject' % entity.eid,
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    77
                                __redirectpath=entity.rest_path()))
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    78
        try:
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    79
            hostconfig = self._cw.execute(
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    80
                'Any X, XC, XH WHERE X cw_host_config_of S, S eid %(s)s, '
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    81
                'X config XC, X match_host XH', {'s': entity.eid})
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    82
        except Unauthorized:
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    83
            pass
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    84
        else:
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    85
            if hostconfig:
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    86
                self.w(u'<h3>%s</h3>' % self._cw._('CWSourceHostConfig_plural'))
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    87
                self._cw.view('editable-table', hostconfig,
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    88
                              displaycols=range(2), w=self.w)
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
    89
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    90
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
    91
MAPPED_SOURCE_TYPES = set( ('pyrorql', 'datafeed') )
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
    92
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    93
class CWSourceMappingTab(EntityView):
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    94
    __regid__ = 'cwsource-mapping'
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    95
    __select__ = (tabs.PrimaryTab.__select__ & is_instance('CWSource')
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
    96
                  & match_user_groups('managers')
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
    97
                  & score_entity(lambda x:x.type in MAPPED_SOURCE_TYPES))
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    98
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    99
    def entity_call(self, entity):
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   100
        _ = self._cw._
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   101
        self.w('<h3>%s</h3>' % _('Entity and relation supported by this source'))
7806
aa30c665bd06 [refactoring] introduce add_etype_button function to properly create button to add a new entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7804
diff changeset
   102
        self.w(add_etype_button(self._cw, 'CWSourceSchemaConfig',
aa30c665bd06 [refactoring] introduce add_etype_button function to properly create button to add a new entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7804
diff changeset
   103
                                __linkto='cw_for_source:%s:subject' % entity.eid))
aa30c665bd06 [refactoring] introduce add_etype_button function to properly create button to add a new entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7804
diff changeset
   104
        self.w(u'<div class="clear"></div>')
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   105
        rset = self._cw.execute(
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   106
            'Any X, SCH, XO ORDERBY ET WHERE X options XO, X cw_for_source S, S eid %(s)s, '
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   107
            'X cw_schema SCH, SCH is ET', {'s': entity.eid})
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   108
        self.wview('table', rset, 'noresult')
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   109
        # self.w('<h3>%s</h3>' % _('Relations that should not be crossed'))
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   110
        # self.w('<p>%s</p>' % _(
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   111
        #     'By default, when a relation is not supported by a source, it is '
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   112
        #     'supposed that a local relation may point to an entity from the '
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   113
        #     'external source. Relations listed here won\'t have this '
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   114
        #     '"crossing" behaviour.'))
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   115
        # self.wview('list', entity.related('cw_dont_cross'), 'noresult')
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   116
        # self.w('<h3>%s</h3>' % _('Relations that can be crossed'))
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   117
        # self.w('<p>%s</p>' % _(
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   118
        #     'By default, when a relation is supported by a source, it is '
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   119
        #     'supposed that a local relation can\'t point to an entity from the '
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   120
        #     'external source. Relations listed here may have this '
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   121
        #     '"crossing" behaviour anyway.'))
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   122
        # self.wview('list', entity.related('cw_may_cross'), 'noresult')
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   123
        checker = MAPPING_CHECKERS.get(entity.type, MappingChecker)(entity)
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   124
        checker.check()
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   125
        if (checker.errors or checker.warnings or checker.infos):
7806
aa30c665bd06 [refactoring] introduce add_etype_button function to properly create button to add a new entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7804
diff changeset
   126
            self.w('<h2>%s</h2>' % _('Detected problems'))
aa30c665bd06 [refactoring] introduce add_etype_button function to properly create button to add a new entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7804
diff changeset
   127
            errors = zip(repeat(_('error')), checker.errors)
aa30c665bd06 [refactoring] introduce add_etype_button function to properly create button to add a new entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7804
diff changeset
   128
            warnings = zip(repeat(_('warning')), checker.warnings)
aa30c665bd06 [refactoring] introduce add_etype_button function to properly create button to add a new entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7804
diff changeset
   129
            infos = zip(repeat(_('warning')), checker.infos)
aa30c665bd06 [refactoring] introduce add_etype_button function to properly create button to add a new entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7804
diff changeset
   130
            self.wview('pyvaltable', pyvalue=chain(errors, warnings, infos))
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   131
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   132
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   133
class MappingChecker(object):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   134
    def __init__(self, cwsource):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   135
        self.cwsource = cwsource
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   136
        self.errors = []
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   137
        self.warnings = []
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   138
        self.infos = []
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   139
        self.schema = cwsource._cw.vreg.schema
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   140
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   141
    def init(self):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   142
        # supported entity types
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   143
        self.sentities = set()
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   144
        # supported relations
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   145
        self.srelations = {}
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   146
        # avoid duplicated messages
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   147
        self.seen = set()
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   148
        # first get mapping as dict/sets
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   149
        for schemacfg in self.cwsource.reverse_cw_for_source:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   150
            self.init_schemacfg(schemacfg)
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   151
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   152
    def init_schemacfg(self, schemacfg):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   153
        cwerschema = schemacfg.schema
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   154
        if cwerschema.__regid__ == 'CWEType':
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   155
            self.sentities.add(cwerschema.name)
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   156
        elif cwerschema.__regid__ == 'CWRType':
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   157
            assert not cwerschema.name in self.srelations
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   158
            self.srelations[cwerschema.name] = None
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   159
        else: # CWAttribute/CWRelation
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   160
            self.srelations.setdefault(cwerschema.rtype.name, []).append(
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   161
                (cwerschema.stype.name, cwerschema.otype.name) )
7375
c8a8fb32733b [cw source ui] fix mapping checking for datafeed source (avoid false positive error, but doesn't check that much things)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7019
diff changeset
   162
            self.sentities.add(cwerschema.stype.name)
c8a8fb32733b [cw source ui] fix mapping checking for datafeed source (avoid false positive error, but doesn't check that much things)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7019
diff changeset
   163
            self.sentities.add(cwerschema.otype.name)
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   164
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   165
    def check(self):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   166
        self.init()
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   167
        error = self.errors.append
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   168
        warning = self.warnings.append
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   169
        info = self.infos.append
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   170
        for etype in self.sentities:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   171
            eschema = self.schema[etype]
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   172
            for rschema, ttypes, role in eschema.relation_definitions():
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   173
                if rschema in META_RTYPES:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   174
                    continue
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   175
                ttypes = [ttype for ttype in ttypes if ttype in self.sentities]
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   176
                if not rschema in self.srelations:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   177
                    for ttype in ttypes:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   178
                        rdef = rschema.role_rdef(etype, ttype, role)
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   179
                        self.seen.add(rdef)
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   180
                        if rdef.role_cardinality(role) in '1+':
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   181
                            error(_('relation %(type)s with %(etype)s as %(role)s '
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   182
                                    'and target type %(target)s is mandatory but '
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   183
                                    'not supported') %
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   184
                                  {'rtype': rschema, 'etype': etype, 'role': role,
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   185
                                   'target': ttype})
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   186
                        elif ttype in self.sentities:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   187
                            warning(_('%s could be supported') % rdef)
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   188
                elif not ttypes:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   189
                    warning(_('relation %(rtype)s with %(etype)s as %(role)s is '
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   190
                              'supported but no target type supported') %
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   191
                            {'rtype': rschema, 'role': role, 'etype': etype})
7375
c8a8fb32733b [cw source ui] fix mapping checking for datafeed source (avoid false positive error, but doesn't check that much things)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7019
diff changeset
   192
        for rtype, rdefs in self.srelations.iteritems():
c8a8fb32733b [cw source ui] fix mapping checking for datafeed source (avoid false positive error, but doesn't check that much things)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7019
diff changeset
   193
            if rdefs is None:
c8a8fb32733b [cw source ui] fix mapping checking for datafeed source (avoid false positive error, but doesn't check that much things)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7019
diff changeset
   194
                rschema = self.schema[rtype]
c8a8fb32733b [cw source ui] fix mapping checking for datafeed source (avoid false positive error, but doesn't check that much things)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7019
diff changeset
   195
                for subj, obj in rschema.rdefs:
c8a8fb32733b [cw source ui] fix mapping checking for datafeed source (avoid false positive error, but doesn't check that much things)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7019
diff changeset
   196
                    if subj in self.sentities and obj in self.sentities:
c8a8fb32733b [cw source ui] fix mapping checking for datafeed source (avoid false positive error, but doesn't check that much things)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7019
diff changeset
   197
                        break
c8a8fb32733b [cw source ui] fix mapping checking for datafeed source (avoid false positive error, but doesn't check that much things)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7019
diff changeset
   198
                else:
c8a8fb32733b [cw source ui] fix mapping checking for datafeed source (avoid false positive error, but doesn't check that much things)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7019
diff changeset
   199
                    error(_('relation %s is supported but none of its definitions '
c8a8fb32733b [cw source ui] fix mapping checking for datafeed source (avoid false positive error, but doesn't check that much things)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7019
diff changeset
   200
                            'matches supported entities') % rtype)
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   201
        self.custom_check()
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   202
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   203
    def custom_check(self):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   204
        pass
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   205
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   206
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   207
class PyroRQLMappingChecker(MappingChecker):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   208
    """pyrorql source mapping checker"""
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   209
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   210
    def init(self):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   211
        self.dontcross = set()
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   212
        self.maycross = set()
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   213
        super(PyroRQLMappingChecker, self).init()
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   214
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   215
    def init_schemacfg(self, schemacfg):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   216
        options = schemacfg.options or ()
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   217
        if 'dontcross' in options:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   218
            self.dontcross.add(schemacfg.schema.name)
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   219
        else:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   220
            super(PyroRQLMappingChecker, self).init_schemacfg(schemacfg)
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   221
            if 'maycross' in options:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   222
                self.maycross.add(schemacfg.schema.name)
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   223
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   224
    def custom_check(self):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   225
        error = self.errors.append
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   226
        info = self.infos.append
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   227
        for etype in self.sentities:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   228
            eschema = self.schema[etype]
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   229
            for rschema, ttypes, role in eschema.relation_definitions():
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   230
                if rschema in META_RTYPES:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   231
                    continue
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   232
                if not rschema in self.srelations:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   233
                    if rschema not in self.dontcross:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   234
                        if role == 'subject' and rschema.inlined:
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   235
                            error(_('inlined relation %(rtype)s of %(etype)s '
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   236
                                    'should be supported') %
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   237
                                  {'rtype': rschema, 'etype': etype})
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   238
                        elif (rschema not in self.seen and rschema not in self.maycross):
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   239
                            info(_('you may want to specify something for %s') %
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   240
                                 rschema)
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   241
                            self.seen.add(rschema)
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   242
                elif rschema in self.maycross and rschema.inlined:
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   243
                    error(_('you should un-inline relation %s which is '
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
   244
                            'supported and may be crossed ') % rschema)
6944
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   245
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   246
MAPPING_CHECKERS = {
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   247
    'pyrorql': PyroRQLMappingChecker,
0cf10429ad39 [sources] rewrite the way pyrorql mapping are stored in the database so it can be reused for other sources (eg datafeed+cwxml)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6724
diff changeset
   248
    }
6962
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   249
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   250
# sources management view ######################################################
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   251
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   252
class ManageSourcesAction(actions.ManagersAction):
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   253
    __regid__ = 'cwsource'
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   254
    title = _('data sources')
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   255
    category = 'manage'
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   256
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   257
class CWSourceManagementView(StartupView):
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   258
    __regid__ = 'cw.source-management'
7455
694b21f0fc62 [views] add columns parser and latest_retrieval to cwsource admin view
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7375
diff changeset
   259
    rql = ('Any S, ST, SP, SD, SN ORDERBY SN WHERE S is CWSource, S name SN, S type ST, '
694b21f0fc62 [views] add columns parser and latest_retrieval to cwsource admin view
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7375
diff changeset
   260
           'S latest_retrieval SD, S parser SP')
6962
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   261
    title = _('data sources management')
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   262
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   263
    def call(self, **kwargs):
220e32f058be [management ui] add views to manage sources similarly to users
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6944
diff changeset
   264
        self.w('<h1>%s</h1>' % self._cw._(self.title))
7852
cb3f700399b2 [schema, cleanup] one more case for add_etype_button
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7847
diff changeset
   265
        self.w(add_etype_button(self._cw, 'CWSource'))
cb3f700399b2 [schema, cleanup] one more case for add_etype_button
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7847
diff changeset
   266
        self.w(u'<div class="clear"></div>')
7455
694b21f0fc62 [views] add columns parser and latest_retrieval to cwsource admin view
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7375
diff changeset
   267
        self.wview('table', self._cw.execute(self.rql), displaycols=range(4))
7832
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
   268
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
   269
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
   270
# breadcrumbs configuration ####################################################
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
   271
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
   272
class CWsourceConfigIBreadCrumbsAdapter(ibreadcrumbs.IBreadCrumbsAdapter):
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
   273
    __select__ = is_instance('CWSourceHostConfig', 'CWSourceSchemaConfig')
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
   274
    def parent_entity(self):
953f224357af [sources management] nicer source management view and breadcrumbs (closes #1946329)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7806
diff changeset
   275
        return self.entity.cwsource