server/migractions.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Tue, 12 Oct 2010 14:13:07 +0200
changeset 6454 97203d0af4cb
parent 6427 c8a5ac2d1eaa
child 6574 a7bbd69f1d1c
permissions -rw-r--r--
[test/possible views] skip deprecated views
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     1
# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
"""a class implementing basic actions used in migration scripts.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
The following schema actions are supported for now:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
* add/drop/rename attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
* add/drop entity/relation type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
* rename entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
The following data actions are supported for now:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
* add an entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
* execute raw RQL queries
5850
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
    28
"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    29
4925
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
    30
from __future__ import with_statement
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
    31
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    32
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    33
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    34
import sys
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    35
import os
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    36
import tarfile
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    37
import tempfile
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    38
import shutil
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    39
import os.path as osp
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 972
diff changeset
    40
from datetime import datetime
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
    41
from glob import glob
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
    42
from copy import copy
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
    43
from warnings import warn
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
    44
from contextlib import contextmanager
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    45
2613
5e19c2bb370e R [all] logilab.common 0.44 provides only deprecated
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2493
diff changeset
    46
from logilab.common.deprecation import deprecated
2107
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    47
from logilab.common.decorators import cached, clear_cache
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
    48
from logilab.common.testlib import mock_object
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    49
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
from yams.constraints import SizeConstraint
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    51
from yams.schema2sql import eschema2sql, rschema2sql
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
    52
from yams.schema import RelationDefinitionSchema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    53
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
    54
from cubicweb import AuthenticationError, ExecutionError
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    55
from cubicweb.selectors import is_instance
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
    56
from cubicweb.schema import (ETYPE_NAME_MAP, META_RTYPES, VIRTUAL_RTYPES,
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
    57
                             PURE_VIRTUAL_RTYPES,
2926
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
    58
                             CubicWebRelationSchema, order_eschemas)
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    59
from cubicweb.cwvreg import CW_EVENT_MANAGER
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    60
from cubicweb.dbapi import get_repository, repo_connect
4021
280c910c8710 move i18n / migration modules from cw.common to cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4017
diff changeset
    61
from cubicweb.migration import MigrationHelper, yes
4925
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
    62
from cubicweb.server.session import hooks_control
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
    63
from cubicweb.server import hook
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    64
try:
2107
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    65
    from cubicweb.server import SOURCE_TYPES, schemaserial as ss
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
    66
    from cubicweb.server.utils import manager_userpasswd
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
    67
    from cubicweb.server.sqlutils import sqlexec, SQL_PREFIX
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    68
except ImportError: # LAX
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    69
    pass
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    70
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    71
class ClearGroupMap(hook.Hook):
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    72
    __regid__ = 'cw.migration.clear_group_mapping'
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    73
    __select__ = hook.Hook.__select__ & is_instance('CWGroup')
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    74
    events = ('after_add_entity', 'after_update_entity',)
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    75
    def __call__(self):
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    76
        clear_cache(self.mih, 'group_mapping')
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    77
        self.mih._synchronized.clear()
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
    78
6205
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    79
    @classmethod
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    80
    def mih_register(cls, repo):
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    81
        # may be already registered in tests (e.g. unittest_migractions at
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    82
        # least)
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    83
        if not cls.__regid__ in repo.vreg['after_add_entity_hooks']:
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    84
            repo.vreg.register(ClearGroupMap)
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    85
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
class ServerMigrationHelper(MigrationHelper):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
    """specific migration helper for server side  migration scripts,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    88
    providind actions related to schema/data migration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    89
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
    def __init__(self, config, schema, interactive=True,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
                 repo=None, cnx=None, verbosity=1, connect=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
        MigrationHelper.__init__(self, config, interactive, verbosity)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    94
        if not interactive:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
            assert cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
            assert repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    97
        if cnx is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    98
            assert repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    99
            self._cnx = cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   100
            self.repo = repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   101
        elif connect:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   102
            self.repo_connect()
5019
72734c210836 [c-c] new server_maintenance hook, called on c-c shell / upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   103
        # no config on shell to a remote instance
5034
4781870e97d9 [maintainance] don't crash if we've no in-memory repository
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5019
diff changeset
   104
        if config is not None and (cnx or connect):
6203
d3dea5f84404 [test] during some tests (unittest_migrations), we should not register ClearGroupMap which is actually already there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6200
diff changeset
   105
            repo = self.repo
5019
72734c210836 [c-c] new server_maintenance hook, called on c-c shell / upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   106
            self.session.data['rebuild-infered'] = False
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   107
            # register a hook to clear our group_mapping cache and the
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   108
            # self._synchronized set when some group is added or updated
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   109
            ClearGroupMap.mih = self
6205
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
   110
            ClearGroupMap.mih_register(repo)
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
   111
            CW_EVENT_MANAGER.bind('after-registry-reload',
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
   112
                                  ClearGroupMap.mih_register, repo)
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   113
            # notify we're starting maintenance (called instead of server_start
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   114
            # which is called on regular start
6203
d3dea5f84404 [test] during some tests (unittest_migrations), we should not register ClearGroupMap which is actually already there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6200
diff changeset
   115
            repo.hm.call_hooks('server_maintenance', repo=repo)
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5034
diff changeset
   116
        if not schema and not getattr(config, 'quick_start', False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   117
            schema = config.load_schema(expand_cubes=True)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   118
        self.fs_schema = schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   119
        self._synchronized = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   120
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   121
    # overriden from base MigrationHelper ######################################
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   122
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   123
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   124
    def repo_connect(self):
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   125
        self.repo = get_repository(method='inmemory', config=self.config)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   126
        return self.repo
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   127
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   128
    def cube_upgraded(self, cube, version):
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   129
        self.cmd_set_property('system.version.%s' % cube.lower(),
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   130
                              unicode(version))
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   131
        self.commit()
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   132
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   133
    def shutdown(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   134
        if self.repo is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
            self.repo.shutdown()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   136
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   137
    def migrate(self, vcconf, toupgrade, options):
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   138
        if not options.fs_only:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   139
            if options.backup_db is None:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   140
                self.backup_database()
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   141
            elif options.backup_db:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   142
                self.backup_database(askconfirm=False)
4925
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
   143
        # disable notification during migration
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
   144
        with hooks_control(self.session, self.session.HOOKS_ALLOW_ALL, 'notification'):
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
   145
            super(ServerMigrationHelper, self).migrate(vcconf, toupgrade, options)
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   146
3715
e3ccadb126d7 [shell] make process_script available throuhg c-c shell / migration script context
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3708
diff changeset
   147
    def cmd_process_script(self, migrscript, funcname=None, *args, **kwargs):
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   148
        try:
6042
df9cafb8062c [migration] commit only sql script migration
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6035
diff changeset
   149
            return super(ServerMigrationHelper, self).cmd_process_script(
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   150
                  migrscript, funcname, *args, **kwargs)
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   151
        except ExecutionError, err:
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   152
            print >> sys.stderr, "-> %s" % err
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   153
        except:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   154
            self.rollback()
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   155
            raise
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   156
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   157
    # Adjust docstring
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   158
    cmd_process_script.__doc__ = MigrationHelper.cmd_process_script.__doc__
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   159
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   160
    # server specific migration methods ########################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   161
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   162
    def backup_database(self, backupfile=None, askconfirm=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   163
        config = self.config
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2489
diff changeset
   164
        repo = self.repo_connect()
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   165
        # paths
3192
93c8fdcd943e windows doesn't like ':' in filesc
Aurelien Campeas
parents: 3120
diff changeset
   166
        timestamp = datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   167
        instbkdir = osp.join(config.appdatahome, 'backup')
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   168
        if not osp.exists(instbkdir):
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   169
            os.makedirs(instbkdir)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   170
        backupfile = backupfile or osp.join(instbkdir, '%s-%s.tar.gz'
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   171
                                            % (config.appid, timestamp))
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   172
        # check backup has to be done
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   173
        if osp.exists(backupfile) and not \
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   174
                self.confirm('Backup file %s exists, overwrite it?' % backupfile):
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   175
            print '-> no backup done.'
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   176
            return
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   177
        elif askconfirm and not self.confirm('Backup %s database?' % config.appid):
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   178
            print '-> no backup done.'
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   179
            return
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   180
        open(backupfile,'w').close() # kinda lock
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   181
        os.chmod(backupfile, 0600)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   182
        # backup
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   183
        tmpdir = tempfile.mkdtemp(dir=instbkdir)
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   184
        try:
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   185
            for source in repo.sources:
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   186
                try:
4904
a5250dafdcac [backup] fix misplaced paren
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4893
diff changeset
   187
                    source.backup(osp.join(tmpdir, source.uri), self.confirm)
4893
15ae9a33a7f2 [db backup] fix name error in backup_to_file: we've to pass .confirm all along the chain as for restore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   188
                except Exception, ex:
15ae9a33a7f2 [db backup] fix name error in backup_to_file: we've to pass .confirm all along the chain as for restore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   189
                    print '-> error trying to backup %s [%s]' % (source.uri, ex)
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   190
                    if not self.confirm('Continue anyway?', default='n'):
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   191
                        raise SystemExit(1)
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   192
                    else:
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   193
                        break
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   194
            else:
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   195
                bkup = tarfile.open(backupfile, 'w|gz')
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   196
                for filename in os.listdir(tmpdir):
4721
8f63691ccb7f pylint style fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   197
                    bkup.add(osp.join(tmpdir, filename), filename)
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   198
                bkup.close()
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   199
                # call hooks
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   200
                repo.hm.call_hooks('server_backup', repo=repo, timestamp=timestamp)
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   201
                # done
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   202
                print '-> backup file',  backupfile
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   203
        finally:
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   204
            shutil.rmtree(tmpdir)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   205
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2489
diff changeset
   206
    def restore_database(self, backupfile, drop=True, systemonly=True,
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2489
diff changeset
   207
                         askconfirm=True):
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   208
        # check
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   209
        if not osp.exists(backupfile):
6323
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   210
            raise ExecutionError("Backup file %s doesn't exist" % backupfile)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   211
        if askconfirm and not self.confirm('Restore %s database from %s ?'
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2926
diff changeset
   212
                                           % (self.config.appid, backupfile)):
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   213
            return
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   214
        # unpack backup
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   215
        tmpdir = tempfile.mkdtemp()
3105
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   216
        try:
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   217
            bkup = tarfile.open(backupfile, 'r|gz')
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   218
        except tarfile.ReadError:
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   219
            # assume restoring old backup
4344
066e7884e57d add source in backup/restore failure message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4339
diff changeset
   220
            shutil.copy(backupfile, osp.join(tmpdir, 'system'))
3105
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   221
        else:
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   222
            for name in bkup.getnames():
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   223
                if name[0] in '/.':
6323
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   224
                    raise ExecutionError('Security check failed, path starts with "/" or "."')
3105
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   225
            bkup.close() # XXX seek error if not close+open !?!
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   226
            bkup = tarfile.open(backupfile, 'r|gz')
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   227
            bkup.extractall(path=tmpdir)
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   228
            bkup.close()
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2926
diff changeset
   229
        self.config.open_connections_pools = False
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2926
diff changeset
   230
        repo = self.repo_connect()
2861
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   231
        for source in repo.sources:
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   232
            if systemonly and source.uri != 'system':
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   233
                continue
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   234
            try:
4195
86dcaf6bb92f closes #601987
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3935
diff changeset
   235
                source.restore(osp.join(tmpdir, source.uri), self.confirm, drop)
2861
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   236
            except Exception, exc:
4344
066e7884e57d add source in backup/restore failure message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4339
diff changeset
   237
                print '-> error trying to restore %s [%s]' % (source.uri, exc)
2861
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   238
                if not self.confirm('Continue anyway?', default='n'):
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   239
                    raise SystemExit(1)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   240
        shutil.rmtree(tmpdir)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   241
        # call hooks
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2926
diff changeset
   242
        repo.open_connections_pools()
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   243
        repo.hm.call_hooks('server_restore', repo=repo, timestamp=backupfile)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   244
        print '-> database restored.'
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   245
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   246
    @property
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   247
    def cnx(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   248
        """lazy connection"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   249
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   250
            return self._cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   251
        except AttributeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   252
            sourcescfg = self.repo.config.sources()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   253
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   254
                login = sourcescfg['admin']['login']
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   255
                pwd = sourcescfg['admin']['password']
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   256
            except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   257
                login, pwd = manager_userpasswd()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   258
            while True:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   259
                try:
3659
993997b4b41d 3.6 update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3589
diff changeset
   260
                    self._cnx = repo_connect(self.repo, login, password=pwd)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   261
                    if not 'managers' in self._cnx.user(self.session).groups:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   262
                        print 'migration need an account in the managers group'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   263
                    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   264
                        break
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   265
                except AuthenticationError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   266
                    print 'wrong user/password'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   267
                except (KeyboardInterrupt, EOFError):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   268
                    print 'aborting...'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   269
                    sys.exit(0)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   270
                try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   271
                    login, pwd = manager_userpasswd()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   272
                except (KeyboardInterrupt, EOFError):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   273
                    print 'aborting...'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   274
                    sys.exit(0)
2571
5c837feca73a oops, this is a method
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2570
diff changeset
   275
            self.session.keep_pool_mode('transaction')
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   276
            self.session.data['rebuild-infered'] = False
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   277
            return self._cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   278
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   279
    @property
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   280
    def session(self):
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   281
        if self.config is not None:
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   282
            session = self.repo._get_session(self.cnx.sessionid)
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   283
            if session.pool is None:
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   284
                session.set_read_security(False)
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   285
                session.set_write_security(False)
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   286
            session.set_pool()
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   287
            return session
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   288
        # no access to session on remote instance
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   289
        return None
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   290
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   291
    def commit(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   292
        if hasattr(self, '_cnx'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   293
            self._cnx.commit()
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   294
        if self.session:
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   295
            self.session.set_pool()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   296
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   297
    def rollback(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   298
        if hasattr(self, '_cnx'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   299
            self._cnx.rollback()
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   300
        if self.session:
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   301
            self.session.set_pool()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   302
6186
8ada7df95877 [migration] set ask_confirm=False by default on rqlexec
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6179
diff changeset
   303
    def rqlexecall(self, rqliter, ask_confirm=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   304
        for rql, kwargs in rqliter:
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   305
            self.rqlexec(rql, kwargs, ask_confirm=ask_confirm)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   306
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   307
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   308
    def _create_context(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   309
        """return a dictionary to use as migration script execution context"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   310
        context = super(ServerMigrationHelper, self)._create_context()
4017
a5b4d4f2a1c7 use commit in migration script instead of checkpoint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4011
diff changeset
   311
        context.update({'commit': self.checkpoint,
4330
cb7b68679501 make rollback available in shell (as commit is)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   312
                        'rollback': self.rollback,
4017
a5b4d4f2a1c7 use commit in migration script instead of checkpoint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4011
diff changeset
   313
                        'checkpoint': deprecated('[3.6] use commit')(self.checkpoint),
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
                        'sql': self.sqlexec,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   315
                        'rql': self.rqlexec,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   316
                        'rqliter': self.rqliter,
3707
78596919ede3 [c-c] fixes for shell w/ pyro instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3700
diff changeset
   317
                        'schema': self.repo.get_schema(),
78596919ede3 [c-c] fixes for shell w/ pyro instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3700
diff changeset
   318
                        'cnx': self.cnx,
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   319
                        'fsschema': self.fs_schema,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   320
                        'session' : self.session,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
                        'repo' : self.repo,
2788
8d3dbe577d3a R put version info in deprecation warnings
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   322
                        'synchronize_schema': deprecated()(self.cmd_sync_schema_props_perms), # 3.4
8d3dbe577d3a R put version info in deprecation warnings
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   323
                        'synchronize_eschema': deprecated()(self.cmd_sync_schema_props_perms), # 3.4
8d3dbe577d3a R put version info in deprecation warnings
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   324
                        'synchronize_rschema': deprecated()(self.cmd_sync_schema_props_perms), # 3.4
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
                        })
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
        return context
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   327
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
    def group_mapping(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   330
        """cached group mapping"""
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   331
        return ss.group_mapping(self._cw)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   332
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   333
    @cached
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   334
    def cstrtype_mapping(self):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   335
        """cached constraint types mapping"""
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   336
        return ss.cstrtype_mapping(self._cw)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   337
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   338
    def exec_event_script(self, event, cubepath=None, funcname=None,
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   339
                          *args, **kwargs):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   340
        if cubepath:
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   341
            apc = osp.join(cubepath, 'migration', '%s.py' % event)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   342
        else:
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   343
            apc = osp.join(self.config.migration_scripts_dir(), '%s.py' % event)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   344
        if osp.exists(apc):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
            if self.config.free_wheel:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2788
diff changeset
   346
                self.cmd_deactivate_verification_hooks()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   347
            self.info('executing %s', apc)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   348
            confirm = self.confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
            execscript_confirm = self.execscript_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   350
            self.confirm = yes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   351
            self.execscript_confirm = yes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   352
            try:
3715
e3ccadb126d7 [shell] make process_script available throuhg c-c shell / migration script context
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3708
diff changeset
   353
                return self.cmd_process_script(apc, funcname, *args, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   354
            finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   355
                self.confirm = confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   356
                self.execscript_confirm = execscript_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   357
                if self.config.free_wheel:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2788
diff changeset
   358
                    self.cmd_reactivate_verification_hooks()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   359
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   360
    def install_custom_sql_scripts(self, directory, driver):
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   361
        for fpath in glob(osp.join(directory, '*.sql.%s' % driver)):
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   362
            newname = osp.basename(fpath).replace('.sql.%s' % driver,
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   363
                                                  '.%s.sql' % driver)
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   364
            warn('[3.5.6] rename %s into %s' % (fpath, newname),
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   365
                 DeprecationWarning)
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   366
            print '-> installing', fpath
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   367
            sqlexec(open(fpath).read(), self.session.system_sql, False,
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   368
                    delimiter=';;')
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   369
        for fpath in glob(osp.join(directory, '*.%s.sql' % driver)):
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   370
            print '-> installing', fpath
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   371
            sqlexec(open(fpath).read(), self.session.system_sql, False,
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   372
                    delimiter=';;')
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   373
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   374
    # schema synchronization internals ########################################
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   375
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   376
    def _synchronize_permissions(self, erschema, teid):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   377
        """permission synchronization for an entity or relation type"""
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   378
        assert teid, erschema
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   379
        if 'update' in erschema.ACTIONS or erschema.final:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   380
            # entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   381
            exprtype = u'ERQLExpression'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   382
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   383
            # relation type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   384
            exprtype = u'RRQLExpression'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
        gm = self.group_mapping()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   386
        confirm = self.verbosity >= 2
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   387
        # * remove possibly deprecated permission (eg in the persistent schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   388
        #   but not in the new schema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   389
        # * synchronize existing expressions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   390
        # * add new groups/expressions
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   391
        for action in erschema.ACTIONS:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   392
            perm = '%s_permission' % action
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   393
            # handle groups
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   394
            newgroups = list(erschema.get_groups(action))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   395
            for geid, gname in self.rqlexec('Any G, GN WHERE T %s G, G name GN, '
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   396
                                            'T eid %%(x)s' % perm, {'x': teid},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   397
                                            ask_confirm=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   398
                if not gname in newgroups:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   399
                    if not confirm or self.confirm('Remove %s permission of %s to %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   400
                                                   % (action, erschema, gname)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
                        self.rqlexec('DELETE T %s G WHERE G eid %%(x)s, T eid %s'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   402
                                     % (perm, teid),
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   403
                                     {'x': geid}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   404
                else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   405
                    newgroups.remove(gname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   406
            for gname in newgroups:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   407
                if not confirm or self.confirm('Grant %s permission of %s to %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   408
                                               % (action, erschema, gname)):
6075
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   409
                    try:
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   410
                        self.rqlexec('SET T %s G WHERE G eid %%(x)s, T eid %s'
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   411
                                     % (perm, teid),
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   412
                                     {'x': gm[gname]}, ask_confirm=False)
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   413
                    except KeyError:
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   414
                        self.error('can grant %s perm to unexistant group %s',
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   415
                                   action, gname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   416
            # handle rql expressions
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   417
            newexprs = dict((expr.expression, expr) for expr in erschema.get_rqlexprs(action))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   418
            for expreid, expression in self.rqlexec('Any E, EX WHERE T %s E, E expression EX, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   419
                                                    'T eid %s' % (perm, teid),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   420
                                                    ask_confirm=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
                if not expression in newexprs:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   422
                    if not confirm or self.confirm('Remove %s expression for %s permission of %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   423
                                                   % (expression, action, erschema)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   424
                        # deleting the relation will delete the expression entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   425
                        self.rqlexec('DELETE T %s E WHERE E eid %%(x)s, T eid %s'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   426
                                     % (perm, teid),
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   427
                                     {'x': expreid}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   428
                else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   429
                    newexprs.pop(expression)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   430
            for expression in newexprs.values():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   431
                expr = expression.expression
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   432
                if not confirm or self.confirm('Add %s expression for %s permission of %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   433
                                               % (expr, action, erschema)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   434
                    self.rqlexec('INSERT RQLExpression X: X exprtype %%(exprtype)s, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   435
                                 'X expression %%(expr)s, X mainvars %%(vars)s, T %s X '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   436
                                 'WHERE T eid %%(x)s' % perm,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   437
                                 {'expr': expr, 'exprtype': exprtype,
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   438
                                  'vars': expression.mainvars, 'x': teid},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   439
                                 ask_confirm=False)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   440
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   441
    def _synchronize_rschema(self, rtype, syncrdefs=True,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   442
                             syncperms=True, syncprops=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   443
        """synchronize properties of the persistent relation schema against its
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   444
        current definition:
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   445
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   446
        * description
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4367
diff changeset
   447
        * symmetric, meta
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   448
        * inlined
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   449
        * relation definitions if `syncrdefs`
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   450
        * permissions if `syncperms`
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   451
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   452
        physical schema changes should be handled by repository's schema hooks
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   453
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   454
        rtype = str(rtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   455
        if rtype in self._synchronized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   456
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   457
        self._synchronized.add(rtype)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   458
        rschema = self.fs_schema.rschema(rtype)
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   459
        reporschema = self.repo.schema.rschema(rtype)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   460
        if syncprops:
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   461
            assert reporschema.eid, reporschema
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   462
            self.rqlexecall(ss.updaterschema2rql(rschema, reporschema.eid),
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   463
                            ask_confirm=self.verbosity>=2)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   464
        if syncrdefs:
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   465
            for subj, obj in rschema.rdefs:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   466
                if (subj, obj) not in reporschema.rdefs:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   467
                    continue
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   468
                if rschema in VIRTUAL_RTYPES:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   469
                    continue
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   470
                self._synchronize_rdef_schema(subj, rschema, obj,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   471
                                              syncprops=syncprops,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   472
                                              syncperms=syncperms)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   473
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   474
    def _synchronize_eschema(self, etype, syncrdefs=True,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   475
                             syncperms=True, syncprops=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   476
        """synchronize properties of the persistent entity schema against
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   477
        its current definition:
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   478
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   479
        * description
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   480
        * internationalizable, fulltextindexed, indexed, meta
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   481
        * relations from/to this entity
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   482
        * __unique_together__
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   483
        * permissions if `syncperms`
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   484
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   485
        etype = str(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   486
        if etype in self._synchronized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   487
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   488
        self._synchronized.add(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   489
        repoeschema = self.repo.schema.eschema(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   490
        try:
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   491
            eschema = self.fs_schema.eschema(etype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   492
        except KeyError:
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   493
            return # XXX somewhat unexpected, no?...
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   494
        if syncprops:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   495
            repospschema = repoeschema.specializes()
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   496
            espschema = eschema.specializes()
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   497
            if repospschema and not espschema:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   498
                self.rqlexec('DELETE X specializes Y WHERE X is CWEType, X name %(x)s',
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   499
                             {'x': str(repoeschema)}, ask_confirm=False)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   500
            elif not repospschema and espschema:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   501
                self.rqlexec('SET X specializes Y WHERE X is CWEType, X name %(x)s, '
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   502
                             'Y is CWEType, Y name %(y)s',
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   503
                             {'x': str(repoeschema), 'y': str(espschema)},
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   504
                             ask_confirm=False)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   505
            self.rqlexecall(ss.updateeschema2rql(eschema, repoeschema.eid),
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   506
                            ask_confirm=self.verbosity >= 2)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   507
        if syncperms:
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   508
            self._synchronize_permissions(eschema, repoeschema.eid)
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   509
        if syncrdefs:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   510
            for rschema, targettypes, role in eschema.relation_definitions(True):
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   511
                if rschema in VIRTUAL_RTYPES:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   512
                    continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   513
                if role == 'subject':
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   514
                    if not rschema in repoeschema.subject_relations():
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   515
                        continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   516
                    subjtypes, objtypes = [etype], targettypes
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   517
                else: # role == 'object'
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   518
                    if not rschema in repoeschema.object_relations():
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   519
                        continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   520
                    subjtypes, objtypes = targettypes, [etype]
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   521
                self._synchronize_rschema(rschema, syncrdefs=False,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   522
                                          syncprops=syncprops, syncperms=syncperms)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   523
                reporschema = self.repo.schema.rschema(rschema)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   524
                for subj in subjtypes:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   525
                    for obj in objtypes:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   526
                        if (subj, obj) not in reporschema.rdefs:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   527
                            continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   528
                        self._synchronize_rdef_schema(subj, rschema, obj,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   529
                                                      syncprops=syncprops, syncperms=syncperms)
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   530
        if syncprops: # need to process __unique_together__ after rdefs were processed
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   531
            repo_unique_together = set([frozenset(ut)
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   532
                                        for ut in repoeschema._unique_together])
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   533
            unique_together = set([frozenset(ut)
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   534
                                   for ut in eschema._unique_together])
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   535
            for ut in repo_unique_together - unique_together:
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   536
                restrictions  = ', '.join(['C relations R%(i)d, '
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   537
                                           'R%(i)d relation_type T%(i)d, '
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   538
                                           'R%(i)d from_entity X, '
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   539
                                           'T%(i)d name %%(T%(i)d)s' % {'i': i,
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   540
                                                                        'col':col}
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   541
                                           for (i, col) in enumerate(ut)])
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   542
                substs = {'etype': etype}
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   543
                for i, col in enumerate(ut):
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   544
                    substs['T%d'%i] = col
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   545
                self.rqlexec('DELETE CWUniqueTogetherConstraint C '
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   546
                             'WHERE C constraint_of E, '
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   547
                             '      E name %%(etype)s,'
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   548
                             '      %s' % restrictions,
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   549
                             substs)
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   550
            for ut in unique_together - repo_unique_together:
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   551
                relations = ', '.join(['C relations R%d' % i
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   552
                                       for (i, col) in enumerate(ut)])
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   553
                restrictions  = ', '.join(['R%(i)d relation_type T%(i)d, '
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   554
                                           'R%(i)d from_entity E, '
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   555
                                           'T%(i)d name %%(T%(i)d)s' % {'i': i,
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   556
                                                                        'col':col}
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   557
                                           for (i, col) in enumerate(ut)])
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   558
                substs = {'etype': etype}
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   559
                for i, col in enumerate(ut):
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   560
                    substs['T%d'%i] = col
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   561
                self.rqlexec('INSERT CWUniqueTogetherConstraint C:'
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   562
                             '       C constraint_of E, '
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   563
                             '       %s '
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   564
                             'WHERE '
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   565
                             '      E name %%(etype)s,'
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   566
                             '      %s' % (relations, restrictions),
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   567
                             substs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   568
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   569
    def _synchronize_rdef_schema(self, subjtype, rtype, objtype,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   570
                                 syncperms=True, syncprops=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   571
        """synchronize properties of the persistent relation definition schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   572
        against its current definition:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   573
        * order and other properties
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   574
        * constraints
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   575
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   576
        subjtype, objtype = str(subjtype), str(objtype)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   577
        rschema = self.fs_schema.rschema(rtype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   578
        reporschema = self.repo.schema.rschema(rschema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   579
        if (subjtype, rschema, objtype) in self._synchronized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   580
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   581
        self._synchronized.add((subjtype, rschema, objtype))
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4367
diff changeset
   582
        if rschema.symmetric:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   583
            self._synchronized.add((objtype, rschema, subjtype))
4949
f4dce73da26b [schema sync] don't try to synchronize infered relation defs, fixing a name error on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4947
diff changeset
   584
        rdef = rschema.rdef(subjtype, objtype)
f4dce73da26b [schema sync] don't try to synchronize infered relation defs, fixing a name error on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4947
diff changeset
   585
        if rdef.infered:
f4dce73da26b [schema sync] don't try to synchronize infered relation defs, fixing a name error on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4947
diff changeset
   586
            return # don't try to synchronize infered relation defs
4950
bca0873c0d6e [schema sync] fix another potential name error on synchronizing rdefs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4949
diff changeset
   587
        repordef = reporschema.rdef(subjtype, objtype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   588
        confirm = self.verbosity >= 2
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   589
        if syncprops:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   590
            # properties
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   591
            self.rqlexecall(ss.updaterdef2rql(rdef, repordef.eid),
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   592
                            ask_confirm=confirm)
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   593
            # constraints
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   594
            newconstraints = list(rdef.constraints)
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   595
            # 1. remove old constraints and update constraints of the same type
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   596
            # NOTE: don't use rschema.constraint_by_type because it may be
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   597
            #       out of sync with newconstraints when multiple
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   598
            #       constraints of the same type are used
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   599
            for cstr in repordef.constraints:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   600
                for newcstr in newconstraints:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   601
                    if newcstr.type() == cstr.type():
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   602
                        break
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   603
                else:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   604
                    newcstr = None
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   605
                if newcstr is None:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   606
                    self.rqlexec('DELETE X constrained_by C WHERE C eid %(x)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   607
                                 {'x': cstr.eid}, ask_confirm=confirm)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   608
                else:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   609
                    newconstraints.remove(newcstr)
4947
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   610
                    value = unicode(newcstr.serialize())
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   611
                    if value != unicode(cstr.serialize()):
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   612
                        self.rqlexec('SET X value %(v)s WHERE X eid %(x)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   613
                                     {'x': cstr.eid, 'v': value},
4947
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   614
                                     ask_confirm=confirm)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   615
            # 2. add new constraints
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   616
            cstrtype_map = self.cstrtype_mapping()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   617
            self.rqlexecall(ss.constraints2rql(cstrtype_map, newconstraints,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   618
                                               repordef.eid),
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   619
                            ask_confirm=confirm)
4188
b3258d2afe04 fix virtual relation detection: erschema is no more a relation type but a relation def in _synchronize_permissions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4099
diff changeset
   620
        if syncperms and not rschema in VIRTUAL_RTYPES:
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   621
            self._synchronize_permissions(rdef, repordef.eid)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   622
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   623
    # base actions ############################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   624
4011
394f853bb653 [migration] write migration instructions for permissions handling on relation definition
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3998
diff changeset
   625
    def checkpoint(self, ask_confirm=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   626
        """checkpoint action"""
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   627
        if not ask_confirm or self.confirm('Commit now ?', shell=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   628
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   629
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   630
    def cmd_add_cube(self, cube, update_database=True):
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   631
        self.cmd_add_cubes( (cube,), update_database)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   632
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   633
    def cmd_add_cubes(self, cubes, update_database=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   634
        """update_database is telling if the database schema should be updated
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   635
        or if only the relevant eproperty should be inserted (for the case where
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2275
diff changeset
   636
        a cube has been extracted from an existing instance, so the
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   637
        cube schema is already in there)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   638
        """
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   639
        newcubes = super(ServerMigrationHelper, self).cmd_add_cubes(cubes)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   640
        if not newcubes:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   641
            return
2107
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   642
        for cube in newcubes:
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   643
            self.cmd_set_property('system.version.'+cube,
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   644
                                  self.config.cube_version(cube))
3836
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   645
            # ensure added cube is in config cubes
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   646
            # XXX worth restoring on error?
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   647
            if not cube in self.config._cubes:
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   648
                self.config._cubes += (cube,)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   649
        if not update_database:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   650
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   651
            return
1036
593df4919845 when reading the schema while adding/removing cubes, read schema in non-strict mode
sylvain.thenault@logilab.fr
parents: 1016
diff changeset
   652
        newcubes_schema = self.config.load_schema(construction_mode='non-strict')
3836
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   653
        # XXX we have to replace fs_schema, used in cmd_add_relation_type
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   654
        # etc. and fsschema of migration script contexts
6217
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6208
diff changeset
   655
        self.fs_schema = newcubes_schema
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6208
diff changeset
   656
        self.update_context('fsschema', self.fs_schema)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   657
        new = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   658
        # execute pre-create files
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   659
        driver = self.repo.system_source.dbdriver
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   660
        for pack in reversed(newcubes):
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   661
            cubedir = self.config.cube_dir(pack)
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   662
            self.install_custom_sql_scripts(osp.join(cubedir, 'schema'), driver)
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   663
            self.exec_event_script('precreate', cubedir)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   664
        # add new entity and relation types
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   665
        for rschema in newcubes_schema.relations():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   666
            if not rschema in self.repo.schema:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   667
                self.cmd_add_relation_type(rschema.type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   668
                new.add(rschema.type)
2926
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   669
        toadd = [eschema for eschema in newcubes_schema.entities()
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   670
                 if not eschema in self.repo.schema]
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   671
        for eschema in order_eschemas(toadd):
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   672
            self.cmd_add_entity_type(eschema.type)
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   673
            new.add(eschema.type)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   674
        # check if attributes has been added to existing entities
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   675
        for rschema in newcubes_schema.relations():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   676
            existingschema = self.repo.schema.rschema(rschema.type)
4099
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   677
            for (fromtype, totype) in rschema.rdefs:
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   678
                if (fromtype, totype) in existingschema.rdefs:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   679
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   680
                # check we should actually add the relation definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   681
                if not (fromtype in new or totype in new or rschema in new):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   682
                    continue
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   683
                self.cmd_add_relation_definition(str(fromtype), rschema.type,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   684
                                                 str(totype))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   685
        # execute post-create files
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   686
        for pack in reversed(newcubes):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   687
            self.exec_event_script('postcreate', self.config.cube_dir(pack))
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   688
            self.commit()
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   689
2124
5a0b02f37b23 set removedeps to False by default, raise an exception instead of a simple assertion for error, more remove_cube tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2122
diff changeset
   690
    def cmd_remove_cube(self, cube, removedeps=False):
2122
4ea13a828513 add removedeps option to remove_cube to control wether a cube's dependencies should be removed as well or not
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2107
diff changeset
   691
        removedcubes = super(ServerMigrationHelper, self).cmd_remove_cube(
4ea13a828513 add removedeps option to remove_cube to control wether a cube's dependencies should be removed as well or not
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2107
diff changeset
   692
            cube, removedeps)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   693
        if not removedcubes:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   694
            return
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   695
        fsschema = self.fs_schema
1036
593df4919845 when reading the schema while adding/removing cubes, read schema in non-strict mode
sylvain.thenault@logilab.fr
parents: 1016
diff changeset
   696
        removedcubes_schema = self.config.load_schema(construction_mode='non-strict')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   697
        reposchema = self.repo.schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   698
        # execute pre-remove files
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   699
        for pack in reversed(removedcubes):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   700
            self.exec_event_script('preremove', self.config.cube_dir(pack))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   701
        # remove cubes'entity and relation types
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   702
        for rschema in fsschema.relations():
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   703
            if not rschema in removedcubes_schema and rschema in reposchema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   704
                self.cmd_drop_relation_type(rschema.type)
2926
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   705
        toremove = [eschema for eschema in fsschema.entities()
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   706
                    if not eschema in removedcubes_schema
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   707
                    and eschema in reposchema]
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   708
        for eschema in reversed(order_eschemas(toremove)):
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   709
            self.cmd_drop_entity_type(eschema.type)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   710
        for rschema in fsschema.relations():
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   711
            if rschema in removedcubes_schema and rschema in reposchema:
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   712
                # check if attributes/relations has been added to entities from
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   713
                # other cubes
4099
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   714
                for fromtype, totype in rschema.rdefs:
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   715
                    if (fromtype, totype) not in removedcubes_schema[rschema.type].rdefs and \
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   716
                           (fromtype, totype) in reposchema[rschema.type].rdefs:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   717
                        self.cmd_drop_relation_definition(
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   718
                            str(fromtype), rschema.type, str(totype))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   719
        # execute post-remove files
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   720
        for cube in reversed(removedcubes):
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   721
            self.exec_event_script('postremove', self.config.cube_dir(cube))
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   722
            self.rqlexec('DELETE CWProperty X WHERE X pkey %(pk)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   723
                         {'pk': u'system.version.'+cube}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   724
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   725
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   726
    # schema migration actions ################################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   727
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   728
    def cmd_add_attribute(self, etype, attrname, attrtype=None, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   729
        """add a new attribute on the given entity type"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   730
        if attrtype is None:
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   731
            rschema = self.fs_schema.rschema(attrname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   732
            attrtype = rschema.objects(etype)[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   733
        self.cmd_add_relation_definition(etype, attrname, attrtype, commit=commit)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   734
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   735
    def cmd_drop_attribute(self, etype, attrname, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   736
        """drop an existing attribute from the given entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   737
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   738
        `attrname` is a string giving the name of the attribute to drop
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   739
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   740
        rschema = self.repo.schema.rschema(attrname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   741
        attrtype = rschema.objects(etype)[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   742
        self.cmd_drop_relation_definition(etype, attrname, attrtype, commit=commit)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   743
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   744
    def cmd_rename_attribute(self, etype, oldname, newname, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   745
        """rename an existing attribute of the given entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   746
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   747
        `oldname` is a string giving the name of the existing attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   748
        `newname` is a string giving the name of the renamed attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   749
        """
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   750
        eschema = self.fs_schema.eschema(etype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   751
        attrtype = eschema.destination(newname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   752
        # have to commit this first step anyway to get the definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   753
        # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   754
        self.cmd_add_attribute(etype, newname, attrtype, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   755
        # skipp NULL values if the attribute is required
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   756
        rql = 'SET X %s VAL WHERE X is %s, X %s VAL' % (newname, etype, oldname)
4633
a85b4361fb22 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4580
diff changeset
   757
        card = eschema.rdef(newname).cardinality[0]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   758
        if card == '1':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   759
            rql += ', NOT X %s NULL' % oldname
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   760
        self.rqlexec(rql, ask_confirm=self.verbosity>=2)
3548
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   761
        # XXX if both attributes fulltext indexed, should skip fti rebuild
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   762
        # XXX if old attribute was fti indexed but not the new one old value
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   763
        # won't be removed from the index (this occurs on other kind of
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   764
        # fulltextindexed change...)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   765
        self.cmd_drop_attribute(etype, oldname, commit=commit)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   766
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   767
    def cmd_add_entity_type(self, etype, auto=True, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   768
        """register a new entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   769
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   770
        in auto mode, automatically register entity's relation where the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   771
        targeted type is known
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   772
        """
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   773
        instschema = self.repo.schema
6189
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6187
diff changeset
   774
        assert not etype in instschema, \
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6187
diff changeset
   775
               '%s already defined in the instance schema' % etype
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   776
        eschema = self.fs_schema.eschema(etype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   777
        confirm = self.verbosity >= 2
4043
39ae94e0c8b8 give group mapping where needed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4042
diff changeset
   778
        groupmap = self.group_mapping()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   779
        cstrtypemap = self.cstrtype_mapping()
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   780
        # register the entity into CWEType
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   781
        execute = self._cw.execute
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   782
        ss.execschemarql(execute, eschema, ss.eschema2rql(eschema, groupmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   783
        # add specializes relation if needed
5293
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   784
        specialized = eschema.specializes()
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   785
        if specialized:
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   786
            try:
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   787
                specialized.eid = instschema[specialized].eid
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   788
            except KeyError:
6323
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   789
                raise ExecutionError('trying to add entity type but parent type is '
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   790
                                     'not yet in the database schema')
5293
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   791
            self.rqlexecall(ss.eschemaspecialize2rql(eschema), ask_confirm=confirm)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   792
        # register entity's attributes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   793
        for rschema, attrschema in eschema.attribute_definitions():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   794
            # ignore those meta relations, they will be automatically added
2617
89c62b855f2e [R schema hooks] rename META_RELATIONS_TYPES into META_RTYPES, use it and other schema consts to avoid errors
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2616
diff changeset
   795
            if rschema.type in META_RTYPES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   796
                continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   797
            if not rschema.type in instschema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   798
                # need to add the relation type and to commit to get it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   799
                # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   800
                self.cmd_add_relation_type(rschema.type, False, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   801
            # register relation definition
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   802
            rdef = self._get_rdef(rschema, eschema, eschema.destination(rschema))
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   803
            ss.execschemarql(execute, rdef, ss.rdef2rql(rdef, cstrtypemap, groupmap),)
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   804
        # take care to newly introduced base class
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   805
        # XXX some part of this should probably be under the "if auto" block
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   806
        for spschema in eschema.specialized_by(recursive=False):
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   807
            try:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   808
                instspschema = instschema[spschema]
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   809
            except KeyError:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   810
                # specialized entity type not in schema, ignore
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   811
                continue
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   812
            if instspschema.specializes() != eschema:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   813
                self.rqlexec('SET D specializes P WHERE D eid %(d)s, P name %(pn)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   814
                             {'d': instspschema.eid, 'pn': eschema.type},
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   815
                             ask_confirm=confirm)
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   816
                for rschema, tschemas, role in spschema.relation_definitions(True):
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   817
                    for tschema in tschemas:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   818
                        if not tschema in instschema:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   819
                            continue
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   820
                        if role == 'subject':
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   821
                            subjschema = spschema
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   822
                            objschema = tschema
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3582
diff changeset
   823
                            if rschema.final and rschema in instspschema.subjrels:
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   824
                                # attribute already set, has_rdef would check if
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   825
                                # it's of the same type, we don't want this so
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   826
                                # simply skip here
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   827
                                continue
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   828
                        elif role == 'object':
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   829
                            subjschema = tschema
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   830
                            objschema = spschema
4633
a85b4361fb22 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4580
diff changeset
   831
                        if (rschema.rdef(subjschema, objschema).infered
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   832
                            or (instschema.has_relation(rschema) and
4633
a85b4361fb22 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4580
diff changeset
   833
                                (subjschema, objschema) in instschema[rschema].rdefs)):
4721
8f63691ccb7f pylint style fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   834
                            continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   835
                        self.cmd_add_relation_definition(
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   836
                            subjschema.type, rschema.type, objschema.type)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   837
        if auto:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   838
            # we have commit here to get relation types actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   839
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   840
            added = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   841
            for rschema in eschema.subject_relations():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   842
                # attribute relation have already been processed and
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   843
                # 'owned_by'/'created_by' will be automatically added
2617
89c62b855f2e [R schema hooks] rename META_RELATIONS_TYPES into META_RTYPES, use it and other schema consts to avoid errors
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2616
diff changeset
   844
                if rschema.final or rschema.type in META_RTYPES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   845
                    continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   846
                rtypeadded = rschema.type in instschema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   847
                for targetschema in rschema.objects(etype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   848
                    # ignore relations where the targeted type is not in the
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2275
diff changeset
   849
                    # current instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   850
                    targettype = targetschema.type
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   851
                    if not targettype in instschema and targettype != etype:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   852
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   853
                    if not rtypeadded:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   854
                        # need to add the relation type and to commit to get it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   855
                        # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   856
                        added.append(rschema.type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   857
                        self.cmd_add_relation_type(rschema.type, False, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   858
                        rtypeadded = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   859
                    # register relation definition
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4367
diff changeset
   860
                    # remember this two avoid adding twice non symmetric relation
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   861
                    # such as "Emailthread forked_from Emailthread"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   862
                    added.append((etype, rschema.type, targettype))
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   863
                    rdef = self._get_rdef(rschema, eschema, targetschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   864
                    ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   865
                                     ss.rdef2rql(rdef, cstrtypemap, groupmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   866
            for rschema in eschema.object_relations():
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   867
                if rschema.type in META_RTYPES:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   868
                    continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   869
                rtypeadded = rschema.type in instschema or rschema.type in added
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   870
                for targetschema in rschema.subjects(etype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   871
                    # ignore relations where the targeted type is not in the
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2275
diff changeset
   872
                    # current instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   873
                    targettype = targetschema.type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   874
                    # don't check targettype != etype since in this case the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   875
                    # relation has already been added as a subject relation
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   876
                    if not targettype in instschema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   877
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   878
                    if not rtypeadded:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   879
                        # need to add the relation type and to commit to get it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   880
                        # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   881
                        self.cmd_add_relation_type(rschema.type, False, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   882
                        rtypeadded = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   883
                    elif (targettype, rschema.type, etype) in added:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   884
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   885
                    # register relation definition
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   886
                    rdef = self._get_rdef(rschema, targetschema, eschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   887
                    ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   888
                                     ss.rdef2rql(rdef, cstrtypemap, groupmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   889
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   890
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   891
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   892
    def cmd_drop_entity_type(self, etype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   893
        """unregister an existing entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   894
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   895
        This will trigger deletion of necessary relation types and definitions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   896
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   897
        # XXX what if we delete an entity type which is specialized by other types
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   898
        # unregister the entity from CWEType
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   899
        self.rqlexec('DELETE CWEType X WHERE X name %(etype)s', {'etype': etype},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   900
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   901
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   902
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   903
6115
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   904
    def cmd_rename_entity_type(self, oldname, newname, attrs=None, commit=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   905
        """rename an existing entity type in the persistent schema
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   906
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   907
        `oldname` is a string giving the name of the existing entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   908
        `newname` is a string giving the name of the renamed entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   909
        """
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   910
        schema = self.repo.schema
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   911
        if newname in schema:
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   912
            assert oldname in ETYPE_NAME_MAP, \
6115
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   913
                   '%s should be mapped to %s in ETYPE_NAME_MAP' % (oldname,
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   914
                                                                    newname)
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   915
            if attrs is None:
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   916
                attrs = ','.join(SQL_PREFIX + rschema.type
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   917
                                 for rschema in schema[newname].subject_relations()
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   918
                                 if (rschema.final or rschema.inlined)
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   919
                                 and not rschema in PURE_VIRTUAL_RTYPES)
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   920
            else:
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   921
                attrs += ('eid', 'creation_date', 'modification_date', 'cwuri')
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   922
                attrs = ','.join(SQL_PREFIX + attr for attr in attrs)
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   923
            self.sqlexec('INSERT INTO %s%s(%s) SELECT %s FROM %s%s' % (
5999
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   924
                SQL_PREFIX, newname, attrs, attrs, SQL_PREFIX, oldname),
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   925
                         ask_confirm=False)
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   926
            # old entity type has not been added to the schema, can't gather it
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   927
            new = schema.eschema(newname)
6117
f3fdfcbda54b cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6116
diff changeset
   928
            oldeid = self.rqlexec('CWEType ET WHERE ET name %(on)s',
f3fdfcbda54b cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6116
diff changeset
   929
                                  {'on': oldname}, ask_confirm=False)[0][0]
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   930
            # backport old type relations to new type
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   931
            # XXX workflows, other relations?
5999
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   932
            for r1, rr1 in [('from_entity', 'to_entity'),
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   933
                            ('to_entity', 'from_entity')]:
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   934
                self.rqlexec('SET X %(r1)s NET WHERE X %(r1)s OET, '
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   935
                             'NOT EXISTS(X2 %(r1)s NET, X relation_type XRT, '
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   936
                             'X2 relation_type XRT, X %(rr1)s XTE, X2 %(rr1)s XTE), '
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   937
                             'OET eid %%(o)s, NET eid %%(n)s' % locals(),
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   938
                             {'o': oldeid, 'n': new.eid}, ask_confirm=False)
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   939
            # backport is / is_instance_of relation to new type
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   940
            for rtype in ('is', 'is_instance_of'):
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   941
                self.sqlexec('UPDATE %s_relation SET eid_to=%s WHERE eid_to=%s'
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   942
                             % (rtype, new.eid, oldeid), ask_confirm=False)
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   943
            # delete relations using SQL to avoid relations content removal
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   944
            # triggered by schema synchronization hooks.
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   945
            session = self.session
6116
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   946
            for rdeftype in ('CWRelation', 'CWAttribute'):
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   947
                thispending = set()
6116
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   948
                for eid, in self.sqlexec('SELECT cw_eid FROM cw_%s '
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   949
                                         'WHERE cw_from_entity=%%(eid)s OR '
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   950
                                         ' cw_to_entity=%%(eid)s' % rdeftype,
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   951
                                         {'eid': oldeid}, ask_confirm=False):
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   952
                    # we should add deleted eids into pending eids else we may
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   953
                    # get some validation error on commit since integrity hooks
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   954
                    # may think some required relation is missing... This also ensure
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   955
                    # repository caches are properly cleanup
6426
541659c39f6a [hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6333
diff changeset
   956
                    CleanupDeletedEidsCacheOp.get_instance(session).add_data(eid)
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   957
                    # and don't forget to remove record from system tables
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   958
                    self.repo.system_source.delete_info(
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   959
                        session, session.entity_from_eid(eid, rdeftype),
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   960
                        'system', None)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   961
                    thispending.add(eid)
6116
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   962
                self.sqlexec('DELETE FROM cw_%s '
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   963
                             'WHERE cw_from_entity=%%(eid)s OR '
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   964
                             'cw_to_entity=%%(eid)s' % rdeftype,
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   965
                             {'eid': oldeid}, ask_confirm=False)
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   966
                # now we have to manually cleanup relations pointing to deleted
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   967
                # entities
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   968
                thiseids = ','.join(str(eid) for eid in thispending)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   969
                for rschema, ttypes, role in schema[rdeftype].relation_definitions():
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   970
                    if rschema.type in VIRTUAL_RTYPES:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   971
                        continue
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   972
                    sqls = []
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   973
                    if role == 'object':
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   974
                        if rschema.inlined:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   975
                            for eschema in ttypes:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   976
                                sqls.append('DELETE FROM cw_%s WHERE cw_%s IN(%%s)'
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   977
                                            % (eschema, rschema))
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   978
                        else:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   979
                            sqls.append('DELETE FROM %s_relation WHERE eid_to IN(%%s)'
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   980
                                        % rschema)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   981
                    elif not rschema.inlined:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   982
                        sqls.append('DELETE FROM %s_relation WHERE eid_from IN(%%s)'
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   983
                                    % rschema)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   984
                    for sql in sqls:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   985
                        self.sqlexec(sql % thiseids, ask_confirm=False)
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   986
            # remove the old type: use rql to propagate deletion
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   987
            self.rqlexec('DELETE CWEType ET WHERE ET name %(on)s', {'on': oldname},
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   988
                         ask_confirm=False)
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   989
        else:
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   990
            self.rqlexec('SET ET name %(newname)s WHERE ET is CWEType, ET name %(on)s',
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   991
                         {'newname' : unicode(newname), 'on' : oldname},
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   992
                         ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   993
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   994
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   995
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   996
    def cmd_add_relation_type(self, rtype, addrdef=True, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   997
        """register a new relation type named `rtype`, as described in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   998
        schema description file.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   999
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1000
        `addrdef` is a boolean value; when True, it will also add all relations
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1001
        of the type just added found in the schema definition file. Note that it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1002
        implies an intermediate "commit" which commits the relation type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1003
        creation (but not the relation definitions themselves, for which
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1004
        committing depends on the `commit` argument value).
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1005
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1006
        """
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1007
        reposchema = self.repo.schema
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
  1008
        rschema = self.fs_schema.rschema(rtype)
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1009
        execute = self._cw.execute
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1010
        # register the relation into CWRType and insert necessary relation
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1011
        # definitions
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1012
        ss.execschemarql(execute, rschema, ss.rschema2rql(rschema, addrdef=False))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1013
        if addrdef:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1014
            self.commit()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1015
            gmap = self.group_mapping()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1016
            cmap = self.cstrtype_mapping()
6073
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1017
            done = set()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1018
            for rdef in rschema.rdefs.itervalues():
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1019
                if not (reposchema.has_entity(rdef.subject)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1020
                        and reposchema.has_entity(rdef.object)):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1021
                    continue
6073
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1022
                # symmetric relations appears twice
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1023
                if (rdef.subject, rdef.object) in done:
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1024
                    continue
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1025
                done.add( (rdef.subject, rdef.object) )
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1026
                self._set_rdef_eid(rdef)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1027
                ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1028
                                 ss.rdef2rql(rdef, cmap, gmap))
2700
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1029
            if rtype in META_RTYPES:
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1030
                # if the relation is in META_RTYPES, ensure we're adding it for
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1031
                # all entity types *in the persistent schema*, not only those in
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1032
                # the fs schema
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1033
                for etype in self.repo.schema.entities():
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1034
                    if not etype in self.fs_schema:
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1035
                        # get sample object type and rproperties
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1036
                        objtypes = rschema.objects()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1037
                        assert len(objtypes) == 1, objtypes
2700
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1038
                        objtype = objtypes[0]
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1039
                        rdef = copy(rschema.rdef(rschema.subjects(objtype)[0], objtype))
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1040
                        rdef.subject = etype
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1041
                        rdef.rtype = self.repo.schema.rschema(rschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1042
                        rdef.object = self.repo.schema.rschema(objtype)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1043
                        ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1044
                                         ss.rdef2rql(rdef, cmap, gmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1045
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1046
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1047
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1048
    def cmd_drop_relation_type(self, rtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1049
        """unregister an existing relation type"""
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1050
        # unregister the relation from CWRType
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1051
        self.rqlexec('DELETE CWRType X WHERE X name %r' % rtype,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1052
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1053
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1054
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1055
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1056
    def cmd_rename_relation(self, oldname, newname, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1057
        """rename an existing relation
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1058
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1059
        `oldname` is a string giving the name of the existing relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1060
        `newname` is a string giving the name of the renamed relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1061
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1062
        self.cmd_add_relation_type(newname, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1063
        self.rqlexec('SET X %s Y WHERE X %s Y' % (newname, oldname),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1064
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1065
        self.cmd_drop_relation_type(oldname, commit=commit)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1066
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1067
    def cmd_add_relation_definition(self, subjtype, rtype, objtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1068
        """register a new relation definition, from its definition found in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1069
        schema definition file
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1070
        """
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
  1071
        rschema = self.fs_schema.rschema(rtype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1072
        if not rtype in self.repo.schema:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1073
            self.cmd_add_relation_type(rtype, addrdef=False, commit=True)
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1074
        execute = self._cw.execute
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1075
        rdef = self._get_rdef(rschema, subjtype, objtype)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1076
        ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1077
                         ss.rdef2rql(rdef, self.cstrtype_mapping(),
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1078
                                     self.group_mapping()))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1079
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1080
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1081
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1082
    def _get_rdef(self, rschema, subjtype, objtype):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1083
        return self._set_rdef_eid(rschema.rdefs[(subjtype, objtype)])
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1084
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1085
    def _set_rdef_eid(self, rdef):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1086
        for attr in ('rtype', 'subject', 'object'):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1087
            schemaobj = getattr(rdef, attr)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1088
            if getattr(schemaobj, 'eid', None) is None:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1089
                schemaobj.eid =  self.repo.schema[schemaobj].eid
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1090
                assert schemaobj.eid is not None
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1091
        return rdef
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1092
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1093
    def cmd_drop_relation_definition(self, subjtype, rtype, objtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1094
        """unregister an existing relation definition"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1095
        rschema = self.repo.schema.rschema(rtype)
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1096
        # unregister the definition from CWAttribute or CWRelation
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3582
diff changeset
  1097
        if rschema.final:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1098
            etype = 'CWAttribute'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1099
        else:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1100
            etype = 'CWRelation'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1101
        rql = ('DELETE %s X WHERE X from_entity FE, FE name "%s",'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1102
               'X relation_type RT, RT name "%s", X to_entity TE, TE name "%s"')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1103
        self.rqlexec(rql % (etype, subjtype, rtype, objtype),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1104
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1105
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1106
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1107
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1108
    def cmd_sync_schema_props_perms(self, ertype=None, syncperms=True,
1409
f4dee84a618f sql attributes bugfix
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 1399
diff changeset
  1109
                                    syncprops=True, syncrdefs=True, commit=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1110
        """synchronize the persistent schema against the current definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1111
        schema.
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1112
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1113
        `ertype` can be :
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1114
        - None, in that case everything will be synced ;
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1115
        - a string, it should be an entity type or
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1116
          a relation type. In that case, only the corresponding
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1117
          entities / relations will be synced ;
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1118
        - an rdef object to synchronize only this specific relation definition
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1119
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1120
        It will synch common stuff between the definition schema and the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1121
        actual persistent schema, it won't add/remove any entity or relation.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1122
        """
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1123
        assert syncperms or syncprops, 'nothing to do'
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1124
        if ertype is not None:
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1125
            if isinstance(ertype, RelationDefinitionSchema):
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1126
                ertype = ertype.as_triple()
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1127
            if isinstance(ertype, (tuple, list)):
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1128
                assert len(ertype) == 3, 'not a relation definition'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
  1129
                self._synchronize_rdef_schema(ertype[0], ertype[1], ertype[2],
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1130
                                              syncperms=syncperms,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1131
                                              syncprops=syncprops)
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1132
            else:
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1133
                erschema = self.repo.schema[ertype]
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1134
                if isinstance(erschema, CubicWebRelationSchema):
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1135
                    self._synchronize_rschema(erschema, syncrdefs=syncrdefs,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1136
                                              syncperms=syncperms,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1137
                                              syncprops=syncprops)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1138
                else:
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1139
                    self._synchronize_eschema(erschema, syncrdefs=syncrdefs,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1140
                                              syncperms=syncperms,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1141
                                              syncprops=syncprops)
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1142
        else:
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1143
            for etype in self.repo.schema.entities():
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1144
                self._synchronize_eschema(etype, syncrdefs=syncrdefs,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1145
                                          syncprops=syncprops, syncperms=syncperms)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1146
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1147
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1148
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1149
    def cmd_change_relation_props(self, subjtype, rtype, objtype,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1150
                                  commit=True, **kwargs):
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1151
        """change some properties of a relation definition
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1152
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1153
        you usually want to use sync_schema_props_perms instead.
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1154
        """
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1155
        assert kwargs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1156
        restriction = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1157
        if subjtype and subjtype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1158
            restriction.append('X from_entity FE, FE name "%s"' % subjtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1159
        if objtype and objtype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1160
            restriction.append('X to_entity TE, TE name "%s"' % objtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1161
        if rtype and rtype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1162
            restriction.append('X relation_type RT, RT name "%s"' % rtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1163
        assert restriction
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1164
        values = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1165
        for k, v in kwargs.items():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1166
            values.append('X %s %%(%s)s' % (k, k))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1167
            if isinstance(v, str):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1168
                kwargs[k] = unicode(v)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1169
        rql = 'SET %s WHERE %s' % (','.join(values), ','.join(restriction))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1170
        self.rqlexec(rql, kwargs, ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1171
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1172
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1173
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1174
    def cmd_set_size_constraint(self, etype, rtype, size, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1175
        """set change size constraint of a string attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1176
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1177
        if size is None any size constraint will be removed.
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1178
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1179
        you usually want to use sync_schema_props_perms instead.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1180
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1181
        oldvalue = None
4695
4aaf87e7f79e 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4650
diff changeset
  1182
        for constr in self.repo.schema.eschema(etype).rdef(rtype).constraints:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1183
            if isinstance(constr, SizeConstraint):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1184
                oldvalue = constr.max
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1185
        if oldvalue == size:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1186
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1187
        if oldvalue is None and not size is None:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1188
            ceid = self.rqlexec('INSERT CWConstraint C: C value %(v)s, C cstrtype CT '
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1189
                                'WHERE CT name "SizeConstraint"',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1190
                                {'v': SizeConstraint(size).serialize()},
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1191
                                ask_confirm=self.verbosity>=2)[0][0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1192
            self.rqlexec('SET X constrained_by C WHERE X from_entity S, X relation_type R, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1193
                         'S name "%s", R name "%s", C eid %s' % (etype, rtype, ceid),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1194
                         ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1195
        elif not oldvalue is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1196
            if not size is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1197
                self.rqlexec('SET C value %%(v)s WHERE X from_entity S, X relation_type R,'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1198
                             'X constrained_by C, C cstrtype CT, CT name "SizeConstraint",'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1199
                             'S name "%s", R name "%s"' % (etype, rtype),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1200
                             {'v': unicode(SizeConstraint(size).serialize())},
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1201
                             ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1202
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1203
                self.rqlexec('DELETE X constrained_by C WHERE X from_entity S, X relation_type R,'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1204
                             'X constrained_by C, C cstrtype CT, CT name "SizeConstraint",'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1205
                             'S name "%s", R name "%s"' % (etype, rtype),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1206
                             ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1207
                # cleanup unused constraints
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1208
                self.rqlexec('DELETE CWConstraint C WHERE NOT X constrained_by C')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1209
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1210
            self.commit()
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1211
3577
067610a0e12f add version number to deprecation messages
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3548
diff changeset
  1212
    @deprecated('[3.2] use sync_schema_props_perms(ertype, syncprops=False)')
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1213
    def cmd_synchronize_permissions(self, ertype, commit=True):
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1214
        self.cmd_sync_schema_props_perms(ertype, syncprops=False, commit=commit)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1215
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1216
    # Workflows handling ######################################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1217
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1218
    def cmd_add_workflow(self, name, wfof, default=True, commit=False,
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1219
                         **kwargs):
3582
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1220
        """
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1221
        create a new workflow and links it to entity types
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1222
         :type name: unicode
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1223
         :param name: name of the workflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1224
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1225
         :type wfof: string or list/tuple of strings
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1226
         :param wfof: entity type(s) having this workflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1227
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1228
         :type default: bool
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1229
         :param default: tells wether this is the default workflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1230
                   for the specified entity type(s); set it to false in
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1231
                   the case of a subworkflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1232
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1233
         :rtype: `Workflow`
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1234
        """
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1235
        wf = self.cmd_create_entity('Workflow', name=unicode(name),
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1236
                                    **kwargs)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1237
        if not isinstance(wfof, (list, tuple)):
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1238
            wfof = (wfof,)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1239
        for etype in wfof:
2956
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1240
            rset = self.rqlexec(
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1241
                'SET X workflow_of ET WHERE X eid %(x)s, ET name %(et)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1242
                {'x': wf.eid, 'et': etype}, ask_confirm=False)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1243
            assert rset, 'unexistant entity type %s' % etype
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1244
            if default:
2956
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1245
                self.rqlexec(
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1246
                    'SET ET default_workflow X WHERE X eid %(x)s, ET name %(et)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1247
                    {'x': wf.eid, 'et': etype}, ask_confirm=False)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1248
        if commit:
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1249
            self.commit()
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1250
        return wf
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1251
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1252
    # XXX remove once cmd_add_[state|transition] are removed
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1253
    def _get_or_create_wf(self, etypes):
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1254
        if not isinstance(etypes, (list, tuple)):
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1255
            etypes = (etypes,)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1256
        rset = self.rqlexec('Workflow X WHERE X workflow_of ET, ET name %(et)s',
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1257
                            {'et': etypes[0]})
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1258
        if rset:
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1259
            return rset.get_entity(0, 0)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1260
        return self.cmd_add_workflow('%s workflow' % ';'.join(etypes), etypes)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1261
4339
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1262
    @deprecated('[3.5] use add_workflow and Workflow.add_state method',
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1263
                stacklevel=3)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1264
    def cmd_add_state(self, name, stateof, initial=False, commit=False, **kwargs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1265
        """method to ease workflow definition: add a state for one or more
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1266
        entity type(s)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1267
        """
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1268
        wf = self._get_or_create_wf(stateof)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1269
        state = wf.add_state(name, initial, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1270
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1271
            self.commit()
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1272
        return state.eid
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1273
4339
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1274
    @deprecated('[3.5] use add_workflow and Workflow.add_transition method',
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1275
                stacklevel=3)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1276
    def cmd_add_transition(self, name, transitionof, fromstates, tostate,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1277
                           requiredgroups=(), conditions=(), commit=False, **kwargs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1278
        """method to ease workflow definition: add a transition for one or more
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1279
        entity type(s), from one or more state and to a single state
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1280
        """
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1281
        wf = self._get_or_create_wf(transitionof)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1282
        tr = wf.add_transition(name, fromstates, tostate, requiredgroups,
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1283
                               conditions, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1284
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1285
            self.commit()
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1286
        return tr.eid
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1287
4339
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1288
    @deprecated('[3.5] use Transition.set_transition_permissions method',
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1289
                stacklevel=3)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1290
    def cmd_set_transition_permissions(self, treid,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1291
                                       requiredgroups=(), conditions=(),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1292
                                       reset=True, commit=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1293
        """set or add (if `reset` is False) groups and conditions for a
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1294
        transition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1295
        """
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1296
        tr = self._cw.entity_from_eid(treid)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1297
        tr.set_transition_permissions(requiredgroups, conditions, reset)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1298
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1299
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1300
5556
9ab2b4c74baf [entity] introduce a new 'adapters' registry
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1301
    @deprecated('[3.5] use iworkflowable.fire_transition("transition") or '
9ab2b4c74baf [entity] introduce a new 'adapters' registry
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1302
                'iworkflowable.change_state("state")', stacklevel=3)
972
1efba3fd1364 set_state command in shell
sylvain.thenault@logilab.fr
parents: 934
diff changeset
  1303
    def cmd_set_state(self, eid, statename, commit=False):
5556
9ab2b4c74baf [entity] introduce a new 'adapters' registry
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1304
        self._cw.entity_from_eid(eid).cw_adapt_to('IWorkflowable').change_state(statename)
972
1efba3fd1364 set_state command in shell
sylvain.thenault@logilab.fr
parents: 934
diff changeset
  1305
        if commit:
1efba3fd1364 set_state command in shell
sylvain.thenault@logilab.fr
parents: 934
diff changeset
  1306
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1307
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1308
    # CWProperty handling ######################################################
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1309
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1310
    def cmd_property_value(self, pkey):
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1311
        rql = 'Any V WHERE X is CWProperty, X pkey %(k)s, X value V'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1312
        rset = self.rqlexec(rql, {'k': pkey}, ask_confirm=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1313
        return rset[0][0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1314
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1315
    def cmd_set_property(self, pkey, value):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1316
        value = unicode(value)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1317
        try:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1318
            prop = self.rqlexec('CWProperty X WHERE X pkey %(k)s', {'k': pkey},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1319
                                ask_confirm=False).get_entity(0, 0)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1320
        except:
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1321
            self.cmd_create_entity('CWProperty', pkey=unicode(pkey), value=value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1322
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1323
            self.rqlexec('SET X value %(v)s WHERE X pkey %(k)s',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1324
                         {'k': pkey, 'v': value}, ask_confirm=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1325
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1326
    # other data migration commands ###########################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1327
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1328
    @property
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1329
    def _cw(self):
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1330
        session = self.session
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1331
        if session is not None:
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1332
            session.set_pool()
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1333
            return session
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1334
        return self.cnx.request()
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1335
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1336
    def cmd_storage_changed(self, etype, attribute):
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1337
        """migrate entities to a custom storage. The new storage is expected to
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1338
        be set, it will be temporarily removed for the migration.
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1339
        """
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1340
        from logilab.common.shellutils import ProgressBar
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1341
        source = self.repo.system_source
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1342
        storage = source.storage(etype, attribute)
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1343
        source.unset_storage(etype, attribute)
5552
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5551
diff changeset
  1344
        rset = self.rqlexec('Any X WHERE X is %s' % etype, ask_confirm=False)
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1345
        pb = ProgressBar(len(rset))
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1346
        for entity in rset.entities():
5551
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1347
            # fill cache. Do not fetch that attribute using the global rql query
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1348
            # since we may exhaust memory doing that....
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1349
            getattr(entity, attribute)
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1350
            storage.migrate_entity(entity, attribute)
5551
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1351
            # remove from entity cache to avoid memory exhaustion
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1352
            del entity[attribute]
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1353
            pb.update()
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1354
        print
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1355
        source.set_storage(etype, attribute, storage)
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1356
5132
260d73ad4f24 [cleaning] simpler implementation of cmd_create_entity
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5043
diff changeset
  1357
    def cmd_create_entity(self, etype, commit=False, **kwargs):
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1358
        """add a new entity of the given type"""
3708
95e8c3a9698a update prototype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3707
diff changeset
  1359
        entity = self._cw.create_entity(etype, **kwargs)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1360
        if commit:
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1361
            self.commit()
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1362
        return entity
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1363
5928
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1364
    def cmd_update_etype_fti_weight(self, etype, weight):
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1365
        if self.repo.system_source.dbdriver == 'postgres':
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1366
            self.sqlexec('UPDATE appears SET weight=%(weight)s '
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1367
                         'FROM entities as X '
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1368
                         'WHERE X.eid=appears.uid AND X.type=%(type)s',
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1369
                         {'type': etype, 'weight': weight}, ask_confirm=False)
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1370
5850
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1371
    def cmd_reindex_entities(self, etypes=None):
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1372
        """force reindexaction of entities of the given types or of all
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1373
        indexable entity types
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1374
        """
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1375
        from cubicweb.server.checkintegrity import reindex_entities
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1376
        reindex_entities(self.repo.schema, self.session, etypes=etypes)
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1377
4367
fa02f7dccfe4 fix deprecation warnings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4344
diff changeset
  1378
    @deprecated('[3.5] use create_entity', stacklevel=3)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1379
    def cmd_add_entity(self, etype, *args, **kwargs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1380
        """add a new entity of the given type"""
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1381
        return self.cmd_create_entity(etype, *args, **kwargs).eid
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1382
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1383
    @contextmanager
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1384
    def cmd_dropped_constraints(self, etype, attrname, cstrtype,
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1385
                                droprequired=False):
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1386
        """context manager to drop constraints temporarily on fs_schema
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1387
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1388
        `cstrtype` should be a constraint class (or a tuple of classes)
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1389
        and will be passed to isinstance directly
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1390
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1391
        For instance::
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1392
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1393
            >>> with dropped_constraints('MyType', 'myattr',
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1394
            ...                          UniqueConstraint, droprequired=True):
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1395
            ...     add_attribute('MyType', 'myattr')
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1396
            ...     # + instructions to fill MyType.myattr column
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1397
            ...
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1398
            >>>
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1399
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1400
        """
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1401
        rdef = self.fs_schema.eschema(etype).rdef(attrname)
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1402
        original_constraints = rdef.constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1403
        # remove constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1404
        rdef.constraints = [cstr for cstr in original_constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1405
                            if not (cstrtype and isinstance(cstr, cstrtype))]
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1406
        if droprequired:
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1407
            original_cardinality = rdef.cardinality
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1408
            rdef.cardinality = '?' + rdef.cardinality[1]
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1409
        yield
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1410
        # restore original constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1411
        rdef.constraints = original_constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1412
        if droprequired:
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1413
            rdef.cardinality = original_cardinality
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1414
        # update repository schema
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1415
        self.cmd_sync_schema_props_perms(rdef, syncperms=False)
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1416
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1417
    def sqlexec(self, sql, args=None, ask_confirm=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1418
        """execute the given sql if confirmed
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1419
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1420
        should only be used for low level stuff undoable with existing higher
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1421
        level actions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1422
        """
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1423
        if not ask_confirm or self.confirm('Execute sql: %s ?' % sql):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1424
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1425
                cu = self.session.system_sql(sql, args)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1426
            except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1427
                ex = sys.exc_info()[1]
6187
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6186
diff changeset
  1428
                if self.confirm('Error: %s\nabort?' % ex, pdb=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1429
                    raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1430
                return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1431
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1432
                return cu.fetchall()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1433
            except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1434
                # no result to fetch
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1435
                return
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1436
4945
356662a6f06c [migration] new build_descr argument to rqlexec on the migration helper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4925
diff changeset
  1437
    def rqlexec(self, rql, kwargs=None, cachekey=None, build_descr=True,
6186
8ada7df95877 [migration] set ask_confirm=False by default on rqlexec
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6179
diff changeset
  1438
                ask_confirm=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1439
        """rql action"""
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1440
        if cachekey is not None:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1441
            warn('[3.8] cachekey is deprecated, you can safely remove this argument',
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1442
                 DeprecationWarning, stacklevel=2)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1443
        if not isinstance(rql, (tuple, list)):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1444
            rql = ( (rql, kwargs), )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1445
        res = None
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1446
        execute = self._cw.execute
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1447
        for rql, kwargs in rql:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1448
            if kwargs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1449
                msg = '%s (%s)' % (rql, kwargs)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1450
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1451
                msg = rql
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1452
            if not ask_confirm or self.confirm('Execute rql: %s ?' % msg):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1453
                try:
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1454
                    res = execute(rql, kwargs, build_descr=build_descr)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1455
                except Exception, ex:
6187
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6186
diff changeset
  1456
                    if self.confirm('Error: %s\nabort?' % ex, pdb=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1457
                        raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1458
        return res
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1459
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1460
    def rqliter(self, rql, kwargs=None, ask_confirm=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1461
        return ForRqlIterator(self, rql, None, ask_confirm)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1462
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1463
    # broken db commands ######################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1464
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1465
    def cmd_change_attribute_type(self, etype, attr, newtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1466
        """low level method to change the type of an entity attribute. This is
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1467
        a quick hack which has some drawback:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1468
        * only works when the old type can be changed to the new type by the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1469
          underlying rdbms (eg using ALTER TABLE)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1470
        * the actual schema won't be updated until next startup
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1471
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1472
        rschema = self.repo.schema.rschema(attr)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1473
        oldtype = rschema.objects(etype)[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1474
        rdefeid = rschema.rproperty(etype, oldtype, 'eid')
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1475
        sql = ("UPDATE CWAttribute "
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1476
               "SET to_entity=(SELECT eid FROM CWEType WHERE name='%s')"
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1477
               "WHERE eid=%s") % (newtype, rdefeid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1478
        self.sqlexec(sql, ask_confirm=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1479
        dbhelper = self.repo.system_source.dbhelper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1480
        sqltype = dbhelper.TYPE_MAPPING[newtype]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1481
        sql = 'ALTER TABLE %s ALTER COLUMN %s TYPE %s' % (etype, attr, sqltype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1482
        self.sqlexec(sql, ask_confirm=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1483
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1484
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1485
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1486
    def cmd_add_entity_type_table(self, etype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1487
        """low level method to create the sql table for an existing entity.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1488
        This may be useful on accidental desync between the repository schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1489
        and a sql database
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1490
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1491
        dbhelper = self.repo.system_source.dbhelper
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
  1492
        tablesql = eschema2sql(dbhelper, self.repo.schema.eschema(etype),
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
  1493
                               prefix=SQL_PREFIX)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1494
        for sql in tablesql.split(';'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1495
            if sql.strip():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1496
                self.sqlexec(sql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1497
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1498
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1499
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1500
    def cmd_add_relation_type_table(self, rtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1501
        """low level method to create the sql table for an existing relation.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1502
        This may be useful on accidental desync between the repository schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1503
        and a sql database
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1504
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1505
        dbhelper = self.repo.system_source.dbhelper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1506
        tablesql = rschema2sql(dbhelper, self.repo.schema.rschema(rtype))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1507
        for sql in tablesql.split(';'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1508
            if sql.strip():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1509
                self.sqlexec(sql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1510
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1511
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1512
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
  1513
    @deprecated("[3.7] use session.disable_hook_categories('integrity')")
4834
b718626a0e60 move hooks activation control on session object, so we can have a per transaction control. Added a new `hooks_control` context manager for usual modification of hooks activation.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4763
diff changeset
  1514
    def cmd_deactivate_verification_hooks(self):
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
  1515
        self.session.disable_hook_categories('integrity')
4834
b718626a0e60 move hooks activation control on session object, so we can have a per transaction control. Added a new `hooks_control` context manager for usual modification of hooks activation.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4763
diff changeset
  1516
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
  1517
    @deprecated("[3.7] use session.enable_hook_categories('integrity')")
4834
b718626a0e60 move hooks activation control on session object, so we can have a per transaction control. Added a new `hooks_control` context manager for usual modification of hooks activation.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4763
diff changeset
  1518
    def cmd_reactivate_verification_hooks(self):
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
  1519
        self.session.enable_hook_categories('integrity')
4834
b718626a0e60 move hooks activation control on session object, so we can have a per transaction control. Added a new `hooks_control` context manager for usual modification of hooks activation.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4763
diff changeset
  1520
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1521
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1522
class ForRqlIterator:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1523
    """specific rql iterator to make the loop skipable"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1524
    def __init__(self, helper, rql, kwargs, ask_confirm):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1525
        self._h = helper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1526
        self.rql = rql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1527
        self.kwargs = kwargs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1528
        self.ask_confirm = ask_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1529
        self._rsetit = None
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1530
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1531
    def __iter__(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1532
        return self
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1533
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1534
    def _get_rset(self):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1535
        rql, kwargs = self.rql, self.kwargs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1536
        if kwargs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1537
            msg = '%s (%s)' % (rql, kwargs)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1538
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1539
            msg = rql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1540
        if self.ask_confirm:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1541
            if not self._h.confirm('Execute rql: %s ?' % msg):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1542
                raise StopIteration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1543
        try:
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1544
            return self._h._cw.execute(rql, kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1545
        except Exception, ex:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1546
            if self._h.confirm('Error: %s\nabort?' % ex):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1547
                raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1548
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1549
                raise StopIteration
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1550
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1551
    def next(self):
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1552
        if self._rsetit is not None:
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1553
            return self._rsetit.next()
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1554
        rset = self._get_rset()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1555
        self._rsetit = iter(rset)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1556
        return self._rsetit.next()
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1557
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1558
    def entities(self):
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1559
        try:
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1560
            rset = self._get_rset()
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1561
        except StopIteration:
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1562
            return []
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1563
        return rset.entities()