server/migractions.py
author Julien Cristau <julien.cristau@logilab.fr>
Wed, 29 Jan 2014 15:50:19 +0100
changeset 9496 e699fbcc9a61
parent 9478 2d7521881d3d
child 9511 241b1232ed7f
permissions -rw-r--r--
[server/repo] Use session.sessionid instead of session.id Silences deprecation warning.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
9162
bf0a575d8da8 [migraction] rename_entity_type simply warn if old entity type isn't in the schema. Closes #3004069
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8807
diff changeset
     1
# copyright 2003-2013 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
"""a class implementing basic actions used in migration scripts.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
The following schema actions are supported for now:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
* add/drop/rename attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
* add/drop entity/relation type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
* rename entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
The following data actions are supported for now:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
* add an entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
* execute raw RQL queries
5850
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
    28
"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    29
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    30
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    31
import sys
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    32
import os
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    33
import tarfile
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    34
import tempfile
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    35
import shutil
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    36
import os.path as osp
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 972
diff changeset
    37
from datetime import datetime
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
    38
from glob import glob
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
    39
from copy import copy
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
    40
from warnings import warn
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
    41
from contextlib import contextmanager
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    42
2613
5e19c2bb370e R [all] logilab.common 0.44 provides only deprecated
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2493
diff changeset
    43
from logilab.common.deprecation import deprecated
2107
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    44
from logilab.common.decorators import cached, clear_cache
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    45
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    46
from yams.constraints import SizeConstraint
9375
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
    47
from yams.schema2sql import eschema2sql, rschema2sql, unique_index_name
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
    48
from yams.schema import RelationDefinitionSchema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    49
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
    50
from cubicweb import CW_SOFTWARE_ROOT, AuthenticationError, ExecutionError
8190
2a3c1b787688 [vreg] move base registry implementation to logilab.common. Closes #1916014
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8181
diff changeset
    51
from cubicweb.predicates import is_instance
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
    52
from cubicweb.schema import (ETYPE_NAME_MAP, META_RTYPES, VIRTUAL_RTYPES,
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
    53
                             PURE_VIRTUAL_RTYPES,
2926
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
    54
                             CubicWebRelationSchema, order_eschemas)
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    55
from cubicweb.cwvreg import CW_EVENT_MANAGER
8673
8ea63a2cc2cc [db-api] rename repo_connect into _repo_connect to mark it private. Closes #2521848
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8669
diff changeset
    56
from cubicweb.dbapi import get_repository, _repo_connect
4021
280c910c8710 move i18n / migration modules from cw.common to cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4017
diff changeset
    57
from cubicweb.migration import MigrationHelper, yes
9468
39b7a91a3f4c [repo] pylint cleanup, mainly of imports, with a bit of style
Julien Cristau <julien.cristau@logilab.fr>
parents: 9463
diff changeset
    58
from cubicweb.server import hook, schemaserial as ss
39b7a91a3f4c [repo] pylint cleanup, mainly of imports, with a bit of style
Julien Cristau <julien.cristau@logilab.fr>
parents: 9463
diff changeset
    59
from cubicweb.server.utils import manager_userpasswd
39b7a91a3f4c [repo] pylint cleanup, mainly of imports, with a bit of style
Julien Cristau <julien.cristau@logilab.fr>
parents: 9463
diff changeset
    60
from cubicweb.server.sqlutils import sqlexec, SQL_PREFIX
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    61
6571
71b446d1cf93 [dependancy] do not use testlib from outside test: recent lgc versions recommends unittest2 while lgc.testlib needs it, and we don't want it to be necessary to run cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6323
diff changeset
    62
71b446d1cf93 [dependancy] do not use testlib from outside test: recent lgc versions recommends unittest2 while lgc.testlib needs it, and we don't want it to be necessary to run cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6323
diff changeset
    63
def mock_object(**params):
71b446d1cf93 [dependancy] do not use testlib from outside test: recent lgc versions recommends unittest2 while lgc.testlib needs it, and we don't want it to be necessary to run cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6323
diff changeset
    64
    return type('Mock', (), params)()
71b446d1cf93 [dependancy] do not use testlib from outside test: recent lgc versions recommends unittest2 while lgc.testlib needs it, and we don't want it to be necessary to run cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6323
diff changeset
    65
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    66
class ClearGroupMap(hook.Hook):
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    67
    __regid__ = 'cw.migration.clear_group_mapping'
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    68
    __select__ = hook.Hook.__select__ & is_instance('CWGroup')
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    69
    events = ('after_add_entity', 'after_update_entity',)
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    70
    def __call__(self):
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    71
        clear_cache(self.mih, 'group_mapping')
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    72
        self.mih._synchronized.clear()
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
    73
6205
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    74
    @classmethod
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    75
    def mih_register(cls, repo):
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    76
        # may be already registered in tests (e.g. unittest_migractions at
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    77
        # least)
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    78
        if not cls.__regid__ in repo.vreg['after_add_entity_hooks']:
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    79
            repo.vreg.register(ClearGroupMap)
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    80
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    81
class ServerMigrationHelper(MigrationHelper):
9175
a7412e884d7b fix typos in docstring, doc and comments
Julien Cristau <julien.cristau@logilab.fr>
parents: 9164
diff changeset
    82
    """specific migration helper for server side migration scripts,
a7412e884d7b fix typos in docstring, doc and comments
Julien Cristau <julien.cristau@logilab.fr>
parents: 9164
diff changeset
    83
    providing actions related to schema/data migration
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    84
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    85
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
    def __init__(self, config, schema, interactive=True,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
                 repo=None, cnx=None, verbosity=1, connect=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    88
        MigrationHelper.__init__(self, config, interactive, verbosity)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    89
        if not interactive:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
            assert cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
            assert repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
        if cnx is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
            assert repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    94
            self._cnx = cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
            self.repo = repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
        elif connect:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    97
            self.repo_connect()
5019
72734c210836 [c-c] new server_maintenance hook, called on c-c shell / upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
    98
        # no config on shell to a remote instance
5034
4781870e97d9 [maintainance] don't crash if we've no in-memory repository
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5019
diff changeset
    99
        if config is not None and (cnx or connect):
6203
d3dea5f84404 [test] during some tests (unittest_migrations), we should not register ClearGroupMap which is actually already there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6200
diff changeset
   100
            repo = self.repo
9080
f0c00b07799a [migraction] do not access session.data directly
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8807
diff changeset
   101
            self.session.set_shared_data('rebuild-infered', False)
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   102
            # register a hook to clear our group_mapping cache and the
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   103
            # self._synchronized set when some group is added or updated
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   104
            ClearGroupMap.mih = self
6205
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
   105
            ClearGroupMap.mih_register(repo)
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
   106
            CW_EVENT_MANAGER.bind('after-registry-reload',
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
   107
                                  ClearGroupMap.mih_register, repo)
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   108
            # notify we're starting maintenance (called instead of server_start
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   109
            # which is called on regular start
6203
d3dea5f84404 [test] during some tests (unittest_migrations), we should not register ClearGroupMap which is actually already there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6200
diff changeset
   110
            repo.hm.call_hooks('server_maintenance', repo=repo)
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5034
diff changeset
   111
        if not schema and not getattr(config, 'quick_start', False):
7797
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   112
            insert_lperms = self.repo.get_versions()['cubicweb'] < (3, 14, 0) and 'localperms' in config.available_cubes()
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   113
            if insert_lperms:
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   114
                cubes = config._cubes
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   115
                config._cubes += ('localperms',)
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   116
            try:
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   117
                schema = config.load_schema(expand_cubes=True)
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   118
            finally:
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   119
                if insert_lperms:
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   120
                    config._cubes = cubes
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   121
        self.fs_schema = schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   122
        self._synchronized = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   123
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   124
    # overriden from base MigrationHelper ######################################
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   125
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   126
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   127
    def repo_connect(self):
8669
62213a34726e [db-api/configuration] simplify db-api and configuration so that all the connection information is in the repository url, closes #2521848
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8645
diff changeset
   128
        self.repo = get_repository(config=self.config)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
        return self.repo
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   130
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   131
    def cube_upgraded(self, cube, version):
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   132
        self.cmd_set_property('system.version.%s' % cube.lower(),
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   133
                              unicode(version))
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   134
        self.commit()
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   135
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
    def shutdown(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
        if self.repo is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   138
            self.repo.shutdown()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   139
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   140
    def migrate(self, vcconf, toupgrade, options):
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   141
        if not options.fs_only:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   142
            if options.backup_db is None:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   143
                self.backup_database()
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   144
            elif options.backup_db:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   145
                self.backup_database(askconfirm=False)
4925
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
   146
        # disable notification during migration
8210
1d1cfc97f6b9 [migration] fire hooks in postcreate by default (closes #1417110)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8190
diff changeset
   147
        with self.session.allow_all_hooks_but('notification'):
4925
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
   148
            super(ServerMigrationHelper, self).migrate(vcconf, toupgrade, options)
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   149
3715
e3ccadb126d7 [shell] make process_script available throuhg c-c shell / migration script context
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3708
diff changeset
   150
    def cmd_process_script(self, migrscript, funcname=None, *args, **kwargs):
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   151
        try:
6042
df9cafb8062c [migration] commit only sql script migration
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6035
diff changeset
   152
            return super(ServerMigrationHelper, self).cmd_process_script(
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   153
                  migrscript, funcname, *args, **kwargs)
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8694
diff changeset
   154
        except ExecutionError as err:
7896
4c954e1e73ef [lint] remove uses of "print >> sys.stderr" (closes #1908571)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7894
diff changeset
   155
            sys.stderr.write("-> %s\n" % err)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   156
        except BaseException:
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   157
            self.rollback()
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   158
            raise
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   159
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   160
    # Adjust docstring
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   161
    cmd_process_script.__doc__ = MigrationHelper.cmd_process_script.__doc__
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   162
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   163
    # server specific migration methods ########################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   164
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   165
    def backup_database(self, backupfile=None, askconfirm=True, format='native'):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   166
        config = self.config
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2489
diff changeset
   167
        repo = self.repo_connect()
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   168
        # paths
3192
93c8fdcd943e windows doesn't like ':' in filesc
Aurelien Campeas
parents: 3120
diff changeset
   169
        timestamp = datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   170
        instbkdir = osp.join(config.appdatahome, 'backup')
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   171
        if not osp.exists(instbkdir):
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   172
            os.makedirs(instbkdir)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   173
        backupfile = backupfile or osp.join(instbkdir, '%s-%s.tar.gz'
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   174
                                            % (config.appid, timestamp))
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   175
        # check backup has to be done
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   176
        if osp.exists(backupfile) and not \
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   177
                self.confirm('Backup file %s exists, overwrite it?' % backupfile):
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   178
            print '-> no backup done.'
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   179
            return
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   180
        elif askconfirm and not self.confirm('Backup %s database?' % config.appid):
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   181
            print '-> no backup done.'
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   182
            return
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   183
        open(backupfile,'w').close() # kinda lock
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   184
        os.chmod(backupfile, 0600)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   185
        # backup
9456
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   186
        source = repo.system_source
6918
9e607157d4cf [cw-ctl] use default `mkdtemp()` behaviour to store backup file internally
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6816
diff changeset
   187
        tmpdir = tempfile.mkdtemp()
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   188
        try:
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   189
            failed = False
9456
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   190
            try:
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   191
                source.backup(osp.join(tmpdir, source.uri), self.confirm, format=format)
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   192
            except Exception as ex:
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   193
                print '-> error trying to backup %s [%s]' % (source.uri, ex)
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   194
                if not self.confirm('Continue anyway?', default='n'):
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   195
                    raise SystemExit(1)
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   196
                else:
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   197
                    failed = True
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   198
            with open(osp.join(tmpdir, 'format.txt'), 'w') as format_file:
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   199
                format_file.write('%s\n' % format)
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   200
            with open(osp.join(tmpdir, 'versions.txt'), 'w') as version_file:
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   201
                versions = repo.get_versions()
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   202
                for cube, version in versions.iteritems():
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   203
                    version_file.write('%s %s\n' % (cube, version))
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   204
            if not failed:
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   205
                bkup = tarfile.open(backupfile, 'w|gz')
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   206
                for filename in os.listdir(tmpdir):
4721
8f63691ccb7f pylint style fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   207
                    bkup.add(osp.join(tmpdir, filename), filename)
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   208
                bkup.close()
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   209
                # call hooks
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   210
                repo.hm.call_hooks('server_backup', repo=repo, timestamp=timestamp)
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   211
                # done
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   212
                print '-> backup file',  backupfile
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   213
        finally:
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   214
            shutil.rmtree(tmpdir)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   215
9457
d5ed6efd6448 [c-c restore] drop useless restore-all option, and related systemonly argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9456
diff changeset
   216
    def restore_database(self, backupfile, drop=True, askconfirm=True, format='native'):
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   217
        # check
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   218
        if not osp.exists(backupfile):
6323
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   219
            raise ExecutionError("Backup file %s doesn't exist" % backupfile)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   220
        if askconfirm and not self.confirm('Restore %s database from %s ?'
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2926
diff changeset
   221
                                           % (self.config.appid, backupfile)):
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   222
            return
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   223
        # unpack backup
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   224
        tmpdir = tempfile.mkdtemp()
3105
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   225
        try:
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   226
            bkup = tarfile.open(backupfile, 'r|gz')
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   227
        except tarfile.ReadError:
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   228
            # assume restoring old backup
4344
066e7884e57d add source in backup/restore failure message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4339
diff changeset
   229
            shutil.copy(backupfile, osp.join(tmpdir, 'system'))
3105
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   230
        else:
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   231
            for name in bkup.getnames():
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   232
                if name[0] in '/.':
6323
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   233
                    raise ExecutionError('Security check failed, path starts with "/" or "."')
3105
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   234
            bkup.close() # XXX seek error if not close+open !?!
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   235
            bkup = tarfile.open(backupfile, 'r|gz')
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   236
            bkup.extractall(path=tmpdir)
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   237
            bkup.close()
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   238
        if osp.isfile(osp.join(tmpdir, 'format.txt')):
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   239
            with open(osp.join(tmpdir, 'format.txt')) as format_file:
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   240
                written_format = format_file.readline().strip()
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   241
                if written_format in ('portable', 'native'):
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   242
                    format = written_format
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   243
        self.config.init_cnxset_pool = False
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2926
diff changeset
   244
        repo = self.repo_connect()
9456
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   245
        source = repo.system_source
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   246
        try:
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   247
            source.restore(osp.join(tmpdir, source.uri), self.confirm, drop, format)
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   248
        except Exception as exc:
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   249
            print '-> error trying to restore %s [%s]' % (source.uri, exc)
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   250
            if not self.confirm('Continue anyway?', default='n'):
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   251
                raise SystemExit(1)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   252
        shutil.rmtree(tmpdir)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   253
        # call hooks
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   254
        repo.init_cnxset_pool()
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   255
        repo.hm.call_hooks('server_restore', repo=repo, timestamp=backupfile)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   256
        print '-> database restored.'
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   257
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   258
    @property
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   259
    def cnx(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   260
        """lazy connection"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   261
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   262
            return self._cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   263
        except AttributeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   264
            try:
9460
a2a0bc984863 [config] cleanup/refactor server sources file values handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9457
diff changeset
   265
                login = self.repo.config.default_admin_config['login']
a2a0bc984863 [config] cleanup/refactor server sources file values handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9457
diff changeset
   266
                pwd = self.repo.config.default_admin_config['password']
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   267
            except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   268
                login, pwd = manager_userpasswd()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   269
            while True:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   270
                try:
8673
8ea63a2cc2cc [db-api] rename repo_connect into _repo_connect to mark it private. Closes #2521848
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8669
diff changeset
   271
                    self._cnx = _repo_connect(self.repo, login, password=pwd)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   272
                    if not 'managers' in self._cnx.user(self.session).groups:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   273
                        print 'migration need an account in the managers group'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   274
                    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   275
                        break
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   276
                except AuthenticationError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   277
                    print 'wrong user/password'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   278
                except (KeyboardInterrupt, EOFError):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   279
                    print 'aborting...'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   280
                    sys.exit(0)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   281
                try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   282
                    login, pwd = manager_userpasswd()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   283
                except (KeyboardInterrupt, EOFError):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   284
                    print 'aborting...'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   285
                    sys.exit(0)
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   286
            self.session.keep_cnxset_mode('transaction')
9080
f0c00b07799a [migraction] do not access session.data directly
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8807
diff changeset
   287
            self.session.set_shared_data('rebuild-infered', False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   288
            return self._cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   289
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
    @property
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   291
    def session(self):
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   292
        if self.config is not None:
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   293
            session = self.repo._get_session(self.cnx.sessionid)
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   294
            if session.cnxset is None:
8807
d9aaad2c52e9 [session] drop useless getter and setter for security
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8794
diff changeset
   295
                session.read_security = False
d9aaad2c52e9 [session] drop useless getter and setter for security
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8794
diff changeset
   296
                session.write_security = False
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   297
            session.set_cnxset()
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   298
            return session
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   299
        # no access to session on remote instance
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   300
        return None
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   301
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   302
    def commit(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   303
        if hasattr(self, '_cnx'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   304
            self._cnx.commit()
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   305
        if self.session:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   306
            self.session.set_cnxset()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   307
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   308
    def rollback(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   309
        if hasattr(self, '_cnx'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   310
            self._cnx.rollback()
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   311
        if self.session:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   312
            self.session.set_cnxset()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   313
6186
8ada7df95877 [migration] set ask_confirm=False by default on rqlexec
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6179
diff changeset
   314
    def rqlexecall(self, rqliter, ask_confirm=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   315
        for rql, kwargs in rqliter:
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   316
            self.rqlexec(rql, kwargs, ask_confirm=ask_confirm)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   317
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   318
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   319
    def _create_context(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   320
        """return a dictionary to use as migration script execution context"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
        context = super(ServerMigrationHelper, self)._create_context()
4017
a5b4d4f2a1c7 use commit in migration script instead of checkpoint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4011
diff changeset
   322
        context.update({'commit': self.checkpoint,
4330
cb7b68679501 make rollback available in shell (as commit is)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   323
                        'rollback': self.rollback,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
                        'sql': self.sqlexec,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
                        'rql': self.rqlexec,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
                        'rqliter': self.rqliter,
3707
78596919ede3 [c-c] fixes for shell w/ pyro instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3700
diff changeset
   327
                        'schema': self.repo.get_schema(),
78596919ede3 [c-c] fixes for shell w/ pyro instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3700
diff changeset
   328
                        'cnx': self.cnx,
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   329
                        'fsschema': self.fs_schema,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   330
                        'session' : self.session,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
                        'repo' : self.repo,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   332
                        })
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   333
        return context
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   334
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   335
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   336
    def group_mapping(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   337
        """cached group mapping"""
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   338
        return ss.group_mapping(self._cw)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   339
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   340
    @cached
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   341
    def cstrtype_mapping(self):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   342
        """cached constraint types mapping"""
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   343
        return ss.cstrtype_mapping(self._cw)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   344
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   345
    def cmd_exec_event_script(self, event, cube=None, funcname=None,
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   346
                              *args, **kwargs):
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   347
        """execute a cube event scripts  `migration/<event>.py` where event
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   348
        is one of 'precreate', 'postcreate', 'preremove' and 'postremove'.
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   349
        """
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   350
        assert event in ('precreate', 'postcreate', 'preremove', 'postremove')
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   351
        if cube:
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   352
            cubepath = self.config.cube_dir(cube)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   353
            apc = osp.join(cubepath, 'migration', '%s.py' % event)
7974
77eec6d6e144 [test] fix regression introduced in 7915:a7f3245e1728 leading to test's postcreate not being executed anymore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7948
diff changeset
   354
        elif kwargs.pop('apphome', False):
77eec6d6e144 [test] fix regression introduced in 7915:a7f3245e1728 leading to test's postcreate not being executed anymore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7948
diff changeset
   355
            apc = osp.join(self.config.apphome, 'migration', '%s.py' % event)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   356
        else:
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   357
            apc = osp.join(self.config.migration_scripts_dir(), '%s.py' % event)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   358
        if osp.exists(apc):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   359
            if self.config.free_wheel:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2788
diff changeset
   360
                self.cmd_deactivate_verification_hooks()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   361
            self.info('executing %s', apc)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   362
            confirm = self.confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   363
            execscript_confirm = self.execscript_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   364
            self.confirm = yes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   365
            self.execscript_confirm = yes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   366
            try:
8210
1d1cfc97f6b9 [migration] fire hooks in postcreate by default (closes #1417110)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8190
diff changeset
   367
                if event == 'postcreate':
1d1cfc97f6b9 [migration] fire hooks in postcreate by default (closes #1417110)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8190
diff changeset
   368
                    with self.session.allow_all_hooks_but():
1d1cfc97f6b9 [migration] fire hooks in postcreate by default (closes #1417110)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8190
diff changeset
   369
                        return self.cmd_process_script(apc, funcname, *args, **kwargs)
3715
e3ccadb126d7 [shell] make process_script available throuhg c-c shell / migration script context
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3708
diff changeset
   370
                return self.cmd_process_script(apc, funcname, *args, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   371
            finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   372
                self.confirm = confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   373
                self.execscript_confirm = execscript_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   374
                if self.config.free_wheel:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2788
diff changeset
   375
                    self.cmd_reactivate_verification_hooks()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   376
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   377
    def cmd_install_custom_sql_scripts(self, cube=None):
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   378
        """install a cube custom sql scripts `schema/*.<driver>.sql` where
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   379
        <driver> depends on the instance main database backend (eg 'postgres',
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   380
        'mysql'...)
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   381
        """
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   382
        driver = self.repo.system_source.dbdriver
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   383
        if cube is None:
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   384
            directory = osp.join(CW_SOFTWARE_ROOT, 'schemas')
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   385
        else:
8047
972360b7677b [instance creation] properly search for sql extension in the schema directory. Closes #2068117
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7974
diff changeset
   386
            directory = osp.join(self.config.cube_dir(cube), 'schema')
7990
a673d1d9a738 [diet] drop pre 3.6 API compatibility (but attempt to keep data cmopatibility). Closes #2017916
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7979
diff changeset
   387
        sql_scripts = glob(osp.join(directory, '*.%s.sql' % driver))
7479
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   388
        for fpath in sql_scripts:
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   389
            print '-> installing', fpath
9421
6b673cb4d3b6 [migractions] Don't silently ignore errors when installing sql procedures
Julien Cristau <julien.cristau@logilab.fr>
parents: 9375
diff changeset
   390
            failed = sqlexec(open(fpath).read(), self.session.system_sql, False,
6b673cb4d3b6 [migractions] Don't silently ignore errors when installing sql procedures
Julien Cristau <julien.cristau@logilab.fr>
parents: 9375
diff changeset
   391
                             delimiter=';;')
6b673cb4d3b6 [migractions] Don't silently ignore errors when installing sql procedures
Julien Cristau <julien.cristau@logilab.fr>
parents: 9375
diff changeset
   392
            if failed:
6b673cb4d3b6 [migractions] Don't silently ignore errors when installing sql procedures
Julien Cristau <julien.cristau@logilab.fr>
parents: 9375
diff changeset
   393
                print '-> ERROR, skipping', fpath
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   394
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   395
    # schema synchronization internals ########################################
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   396
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   397
    def _synchronize_permissions(self, erschema, teid):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   398
        """permission synchronization for an entity or relation type"""
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   399
        assert teid, erschema
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   400
        if 'update' in erschema.ACTIONS or erschema.final:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
            # entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   402
            exprtype = u'ERQLExpression'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   403
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   404
            # relation type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   405
            exprtype = u'RRQLExpression'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   406
        gm = self.group_mapping()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   407
        confirm = self.verbosity >= 2
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   408
        # * remove possibly deprecated permission (eg in the persistent schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   409
        #   but not in the new schema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   410
        # * synchronize existing expressions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   411
        # * add new groups/expressions
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   412
        for action in erschema.ACTIONS:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   413
            perm = '%s_permission' % action
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   414
            # handle groups
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   415
            newgroups = list(erschema.get_groups(action))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   416
            for geid, gname in self.rqlexec('Any G, GN WHERE T %s G, G name GN, '
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   417
                                            'T eid %%(x)s' % perm, {'x': teid},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   418
                                            ask_confirm=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   419
                if not gname in newgroups:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   420
                    if not confirm or self.confirm('Remove %s permission of %s to %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   421
                                                   % (action, erschema, gname)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   422
                        self.rqlexec('DELETE T %s G WHERE G eid %%(x)s, T eid %s'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
                                     % (perm, teid),
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   424
                                     {'x': geid}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   425
                else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   426
                    newgroups.remove(gname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   427
            for gname in newgroups:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   428
                if not confirm or self.confirm('Grant %s permission of %s to %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   429
                                               % (action, erschema, gname)):
6075
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   430
                    try:
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   431
                        self.rqlexec('SET T %s G WHERE G eid %%(x)s, T eid %s'
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   432
                                     % (perm, teid),
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   433
                                     {'x': gm[gname]}, ask_confirm=False)
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   434
                    except KeyError:
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   435
                        self.error('can grant %s perm to unexistant group %s',
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   436
                                   action, gname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   437
            # handle rql expressions
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   438
            newexprs = dict((expr.expression, expr) for expr in erschema.get_rqlexprs(action))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   439
            for expreid, expression in self.rqlexec('Any E, EX WHERE T %s E, E expression EX, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   440
                                                    'T eid %s' % (perm, teid),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   441
                                                    ask_confirm=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   442
                if not expression in newexprs:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   443
                    if not confirm or self.confirm('Remove %s expression for %s permission of %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   444
                                                   % (expression, action, erschema)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   445
                        # deleting the relation will delete the expression entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   446
                        self.rqlexec('DELETE T %s E WHERE E eid %%(x)s, T eid %s'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   447
                                     % (perm, teid),
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   448
                                     {'x': expreid}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   449
                else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   450
                    newexprs.pop(expression)
8696
0bb18407c053 [toward py3k] rewrite dict.keys() and dict.values() (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8695
diff changeset
   451
            for expression in newexprs.itervalues():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   452
                expr = expression.expression
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   453
                if not confirm or self.confirm('Add %s expression for %s permission of %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   454
                                               % (expr, action, erschema)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   455
                    self.rqlexec('INSERT RQLExpression X: X exprtype %%(exprtype)s, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   456
                                 'X expression %%(expr)s, X mainvars %%(vars)s, T %s X '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   457
                                 'WHERE T eid %%(x)s' % perm,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   458
                                 {'expr': expr, 'exprtype': exprtype,
7161
e3f69df8dac7 fix failing tests before 3.12 release
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7116
diff changeset
   459
                                  'vars': u','.join(sorted(expression.mainvars)),
e3f69df8dac7 fix failing tests before 3.12 release
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7116
diff changeset
   460
                                  'x': teid},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   461
                                 ask_confirm=False)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   462
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   463
    def _synchronize_rschema(self, rtype, syncrdefs=True,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   464
                             syncperms=True, syncprops=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   465
        """synchronize properties of the persistent relation schema against its
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   466
        current definition:
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   467
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   468
        * description
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4367
diff changeset
   469
        * symmetric, meta
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   470
        * inlined
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   471
        * relation definitions if `syncrdefs`
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   472
        * permissions if `syncperms`
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   473
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   474
        physical schema changes should be handled by repository's schema hooks
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   475
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   476
        rtype = str(rtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   477
        if rtype in self._synchronized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   478
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   479
        self._synchronized.add(rtype)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   480
        rschema = self.fs_schema.rschema(rtype)
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   481
        reporschema = self.repo.schema.rschema(rtype)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   482
        if syncprops:
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   483
            assert reporschema.eid, reporschema
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   484
            self.rqlexecall(ss.updaterschema2rql(rschema, reporschema.eid),
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   485
                            ask_confirm=self.verbosity>=2)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   486
        if syncrdefs:
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   487
            for subj, obj in rschema.rdefs:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   488
                if (subj, obj) not in reporschema.rdefs:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   489
                    continue
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   490
                if rschema in VIRTUAL_RTYPES:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   491
                    continue
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   492
                self._synchronize_rdef_schema(subj, rschema, obj,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   493
                                              syncprops=syncprops,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   494
                                              syncperms=syncperms)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   495
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   496
    def _synchronize_eschema(self, etype, syncrdefs=True,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   497
                             syncperms=True, syncprops=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   498
        """synchronize properties of the persistent entity schema against
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   499
        its current definition:
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   500
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   501
        * description
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   502
        * internationalizable, fulltextindexed, indexed, meta
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   503
        * relations from/to this entity
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   504
        * __unique_together__
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   505
        * permissions if `syncperms`
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   506
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   507
        etype = str(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   508
        if etype in self._synchronized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   509
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   510
        self._synchronized.add(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   511
        repoeschema = self.repo.schema.eschema(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   512
        try:
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   513
            eschema = self.fs_schema.eschema(etype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   514
        except KeyError:
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   515
            return # XXX somewhat unexpected, no?...
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   516
        if syncprops:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   517
            repospschema = repoeschema.specializes()
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   518
            espschema = eschema.specializes()
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   519
            if repospschema and not espschema:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   520
                self.rqlexec('DELETE X specializes Y WHERE X is CWEType, X name %(x)s',
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   521
                             {'x': str(repoeschema)}, ask_confirm=False)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   522
            elif not repospschema and espschema:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   523
                self.rqlexec('SET X specializes Y WHERE X is CWEType, X name %(x)s, '
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   524
                             'Y is CWEType, Y name %(y)s',
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   525
                             {'x': str(repoeschema), 'y': str(espschema)},
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   526
                             ask_confirm=False)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   527
            self.rqlexecall(ss.updateeschema2rql(eschema, repoeschema.eid),
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   528
                            ask_confirm=self.verbosity >= 2)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   529
        if syncperms:
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   530
            self._synchronize_permissions(eschema, repoeschema.eid)
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   531
        if syncrdefs:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   532
            for rschema, targettypes, role in eschema.relation_definitions(True):
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   533
                if rschema in VIRTUAL_RTYPES:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   534
                    continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   535
                if role == 'subject':
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   536
                    if not rschema in repoeschema.subject_relations():
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   537
                        continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   538
                    subjtypes, objtypes = [etype], targettypes
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   539
                else: # role == 'object'
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   540
                    if not rschema in repoeschema.object_relations():
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   541
                        continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   542
                    subjtypes, objtypes = targettypes, [etype]
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   543
                self._synchronize_rschema(rschema, syncrdefs=False,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   544
                                          syncprops=syncprops, syncperms=syncperms)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   545
                reporschema = self.repo.schema.rschema(rschema)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   546
                for subj in subjtypes:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   547
                    for obj in objtypes:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   548
                        if (subj, obj) not in reporschema.rdefs:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   549
                            continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   550
                        self._synchronize_rdef_schema(subj, rschema, obj,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   551
                                                      syncprops=syncprops, syncperms=syncperms)
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   552
        if syncprops: # need to process __unique_together__ after rdefs were processed
9375
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   553
            # mappings from constraint name to columns
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   554
            # filesystem (fs) and repository (repo) wise
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   555
            fs = {}
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   556
            repo = {}
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   557
            for cols in eschema._unique_together or ():
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   558
                fs[unique_index_name(repoeschema, cols)] = sorted(cols)
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   559
            schemaentity = self.session.entity_from_eid(repoeschema.eid)
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   560
            for entity in schemaentity.related('constraint_of', 'object',
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   561
                                               targettypes=('CWUniqueTogetherConstraint',)).entities():
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   562
                repo[entity.name] = sorted(rel.name for rel in entity.relations)
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   563
            added = set(fs) - set(repo)
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   564
            removed = set(repo) - set(fs)
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   565
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   566
            for name in removed:
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   567
                self.rqlexec('DELETE CWUniqueTogetherConstraint C WHERE C name %(name)s',
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   568
                             {'name': name})
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   569
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   570
            def possible_unique_constraint(cols):
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   571
                for name in cols:
9195
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   572
                    rschema = repoeschema.subjrels.get(name)
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   573
                    if rschema is None:
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   574
                        print 'dont add %s unique constraint on %s, missing %s' % (
9375
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   575
                            ','.join(cols), eschema, name)
9195
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   576
                        return False
9241
cbee712dd310 [migractions] rschema.final.inlined -> rschema.inlined
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9195
diff changeset
   577
                    if not (rschema.final or rschema.inlined):
9195
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   578
                        print 'dont add %s unique constraint on %s, %s is neither final nor inlined' % (
9375
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   579
                            ','.join(cols), eschema, name)
9195
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   580
                        return False
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   581
                return True
9375
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   582
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   583
            for name in added:
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   584
                if possible_unique_constraint(fs[name]):
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   585
                    rql, substs = ss._uniquetogether2rql(eschema, fs[name])
9195
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   586
                    substs['x'] = repoeschema.eid
9375
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   587
                    substs['name'] = name
9195
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   588
                    self.rqlexec(rql, substs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   589
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   590
    def _synchronize_rdef_schema(self, subjtype, rtype, objtype,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   591
                                 syncperms=True, syncprops=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   592
        """synchronize properties of the persistent relation definition schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   593
        against its current definition:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   594
        * order and other properties
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   595
        * constraints
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   596
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   597
        subjtype, objtype = str(subjtype), str(objtype)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   598
        rschema = self.fs_schema.rschema(rtype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   599
        reporschema = self.repo.schema.rschema(rschema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   600
        if (subjtype, rschema, objtype) in self._synchronized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   601
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   602
        self._synchronized.add((subjtype, rschema, objtype))
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4367
diff changeset
   603
        if rschema.symmetric:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   604
            self._synchronized.add((objtype, rschema, subjtype))
4949
f4dce73da26b [schema sync] don't try to synchronize infered relation defs, fixing a name error on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4947
diff changeset
   605
        rdef = rschema.rdef(subjtype, objtype)
f4dce73da26b [schema sync] don't try to synchronize infered relation defs, fixing a name error on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4947
diff changeset
   606
        if rdef.infered:
f4dce73da26b [schema sync] don't try to synchronize infered relation defs, fixing a name error on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4947
diff changeset
   607
            return # don't try to synchronize infered relation defs
4950
bca0873c0d6e [schema sync] fix another potential name error on synchronizing rdefs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4949
diff changeset
   608
        repordef = reporschema.rdef(subjtype, objtype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   609
        confirm = self.verbosity >= 2
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   610
        if syncprops:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   611
            # properties
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   612
            self.rqlexecall(ss.updaterdef2rql(rdef, repordef.eid),
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   613
                            ask_confirm=confirm)
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   614
            # constraints
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   615
            newconstraints = list(rdef.constraints)
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   616
            # 1. remove old constraints and update constraints of the same type
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   617
            # NOTE: don't use rschema.constraint_by_type because it may be
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   618
            #       out of sync with newconstraints when multiple
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   619
            #       constraints of the same type are used
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   620
            for cstr in repordef.constraints:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   621
                for newcstr in newconstraints:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   622
                    if newcstr.type() == cstr.type():
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   623
                        break
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   624
                else:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   625
                    newcstr = None
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   626
                if newcstr is None:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   627
                    self.rqlexec('DELETE X constrained_by C WHERE C eid %(x)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   628
                                 {'x': cstr.eid}, ask_confirm=confirm)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   629
                else:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   630
                    newconstraints.remove(newcstr)
4947
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   631
                    value = unicode(newcstr.serialize())
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   632
                    if value != unicode(cstr.serialize()):
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   633
                        self.rqlexec('SET X value %(v)s WHERE X eid %(x)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   634
                                     {'x': cstr.eid, 'v': value},
4947
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   635
                                     ask_confirm=confirm)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   636
            # 2. add new constraints
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   637
            cstrtype_map = self.cstrtype_mapping()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   638
            self.rqlexecall(ss.constraints2rql(cstrtype_map, newconstraints,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   639
                                               repordef.eid),
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   640
                            ask_confirm=confirm)
4188
b3258d2afe04 fix virtual relation detection: erschema is no more a relation type but a relation def in _synchronize_permissions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4099
diff changeset
   641
        if syncperms and not rschema in VIRTUAL_RTYPES:
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   642
            self._synchronize_permissions(rdef, repordef.eid)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   643
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   644
    # base actions ############################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   645
4011
394f853bb653 [migration] write migration instructions for permissions handling on relation definition
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3998
diff changeset
   646
    def checkpoint(self, ask_confirm=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   647
        """checkpoint action"""
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   648
        if not ask_confirm or self.confirm('Commit now ?', shell=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   649
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   650
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   651
    def cmd_add_cube(self, cube, update_database=True):
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   652
        self.cmd_add_cubes( (cube,), update_database)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   653
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   654
    def cmd_add_cubes(self, cubes, update_database=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   655
        """update_database is telling if the database schema should be updated
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   656
        or if only the relevant eproperty should be inserted (for the case where
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2275
diff changeset
   657
        a cube has been extracted from an existing instance, so the
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   658
        cube schema is already in there)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   659
        """
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   660
        newcubes = super(ServerMigrationHelper, self).cmd_add_cubes(cubes)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   661
        if not newcubes:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   662
            return
2107
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   663
        for cube in newcubes:
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   664
            self.cmd_set_property('system.version.'+cube,
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   665
                                  self.config.cube_version(cube))
3836
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   666
            # ensure added cube is in config cubes
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   667
            # XXX worth restoring on error?
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   668
            if not cube in self.config._cubes:
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   669
                self.config._cubes += (cube,)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   670
        if not update_database:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   671
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   672
            return
1036
593df4919845 when reading the schema while adding/removing cubes, read schema in non-strict mode
sylvain.thenault@logilab.fr
parents: 1016
diff changeset
   673
        newcubes_schema = self.config.load_schema(construction_mode='non-strict')
3836
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   674
        # XXX we have to replace fs_schema, used in cmd_add_relation_type
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   675
        # etc. and fsschema of migration script contexts
6217
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6208
diff changeset
   676
        self.fs_schema = newcubes_schema
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6208
diff changeset
   677
        self.update_context('fsschema', self.fs_schema)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   678
        new = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   679
        # execute pre-create files
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   680
        driver = self.repo.system_source.dbdriver
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   681
        for cube in reversed(newcubes):
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   682
            self.cmd_install_custom_sql_scripts(cube)
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   683
            self.cmd_exec_event_script('precreate', cube)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   684
        # add new entity and relation types
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   685
        for rschema in newcubes_schema.relations():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   686
            if not rschema in self.repo.schema:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   687
                self.cmd_add_relation_type(rschema.type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   688
                new.add(rschema.type)
2926
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   689
        toadd = [eschema for eschema in newcubes_schema.entities()
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   690
                 if not eschema in self.repo.schema]
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   691
        for eschema in order_eschemas(toadd):
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   692
            self.cmd_add_entity_type(eschema.type)
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   693
            new.add(eschema.type)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   694
        # check if attributes has been added to existing entities
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   695
        for rschema in newcubes_schema.relations():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   696
            existingschema = self.repo.schema.rschema(rschema.type)
4099
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   697
            for (fromtype, totype) in rschema.rdefs:
9164
4415209bea0a [migration] when adding a cube, skip infered relations (closes #3005576)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9162
diff changeset
   698
                # if rdef already exists or is infered from inheritance,
4415209bea0a [migration] when adding a cube, skip infered relations (closes #3005576)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9162
diff changeset
   699
                # don't add it
4415209bea0a [migration] when adding a cube, skip infered relations (closes #3005576)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9162
diff changeset
   700
                if (fromtype, totype) in existingschema.rdefs \
4415209bea0a [migration] when adding a cube, skip infered relations (closes #3005576)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9162
diff changeset
   701
                        or rschema.rdefs[(fromtype, totype)].infered:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   702
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   703
                # check we should actually add the relation definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   704
                if not (fromtype in new or totype in new or rschema in new):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   705
                    continue
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   706
                self.cmd_add_relation_definition(str(fromtype), rschema.type,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   707
                                                 str(totype))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   708
        # execute post-create files
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   709
        for cube in reversed(newcubes):
8210
1d1cfc97f6b9 [migration] fire hooks in postcreate by default (closes #1417110)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8190
diff changeset
   710
            with self.session.allow_all_hooks_but():
1d1cfc97f6b9 [migration] fire hooks in postcreate by default (closes #1417110)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8190
diff changeset
   711
                self.cmd_exec_event_script('postcreate', cube)
1d1cfc97f6b9 [migration] fire hooks in postcreate by default (closes #1417110)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8190
diff changeset
   712
                self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   713
2124
5a0b02f37b23 set removedeps to False by default, raise an exception instead of a simple assertion for error, more remove_cube tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2122
diff changeset
   714
    def cmd_remove_cube(self, cube, removedeps=False):
2122
4ea13a828513 add removedeps option to remove_cube to control wether a cube's dependencies should be removed as well or not
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2107
diff changeset
   715
        removedcubes = super(ServerMigrationHelper, self).cmd_remove_cube(
4ea13a828513 add removedeps option to remove_cube to control wether a cube's dependencies should be removed as well or not
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2107
diff changeset
   716
            cube, removedeps)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   717
        if not removedcubes:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   718
            return
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   719
        fsschema = self.fs_schema
1036
593df4919845 when reading the schema while adding/removing cubes, read schema in non-strict mode
sylvain.thenault@logilab.fr
parents: 1016
diff changeset
   720
        removedcubes_schema = self.config.load_schema(construction_mode='non-strict')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   721
        reposchema = self.repo.schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   722
        # execute pre-remove files
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   723
        for cube in reversed(removedcubes):
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   724
            self.cmd_exec_event_script('preremove', cube)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   725
        # remove cubes'entity and relation types
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   726
        for rschema in fsschema.relations():
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   727
            if not rschema in removedcubes_schema and rschema in reposchema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   728
                self.cmd_drop_relation_type(rschema.type)
2926
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   729
        toremove = [eschema for eschema in fsschema.entities()
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   730
                    if not eschema in removedcubes_schema
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   731
                    and eschema in reposchema]
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   732
        for eschema in reversed(order_eschemas(toremove)):
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   733
            self.cmd_drop_entity_type(eschema.type)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   734
        for rschema in fsschema.relations():
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   735
            if rschema in removedcubes_schema and rschema in reposchema:
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   736
                # check if attributes/relations has been added to entities from
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   737
                # other cubes
4099
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   738
                for fromtype, totype in rschema.rdefs:
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   739
                    if (fromtype, totype) not in removedcubes_schema[rschema.type].rdefs and \
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   740
                           (fromtype, totype) in reposchema[rschema.type].rdefs:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   741
                        self.cmd_drop_relation_definition(
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   742
                            str(fromtype), rschema.type, str(totype))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   743
        # execute post-remove files
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   744
        for cube in reversed(removedcubes):
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   745
            self.cmd_exec_event_script('postremove', cube)
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   746
            self.rqlexec('DELETE CWProperty X WHERE X pkey %(pk)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   747
                         {'pk': u'system.version.'+cube}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   748
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   749
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   750
    # schema migration actions ################################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   751
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   752
    def cmd_add_attribute(self, etype, attrname, attrtype=None, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   753
        """add a new attribute on the given entity type"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   754
        if attrtype is None:
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   755
            rschema = self.fs_schema.rschema(attrname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   756
            attrtype = rschema.objects(etype)[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   757
        self.cmd_add_relation_definition(etype, attrname, attrtype, commit=commit)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   758
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   759
    def cmd_drop_attribute(self, etype, attrname, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   760
        """drop an existing attribute from the given entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   761
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   762
        `attrname` is a string giving the name of the attribute to drop
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   763
        """
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   764
        try:
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   765
            rschema = self.repo.schema.rschema(attrname)
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   766
            attrtype = rschema.objects(etype)[0]
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   767
        except KeyError:
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   768
            print 'warning: attribute %s %s is not known, skip deletion' % (
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   769
                etype, attrname)
7558
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   770
        else:
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   771
            self.cmd_drop_relation_definition(etype, attrname, attrtype,
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   772
                                              commit=commit)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   773
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   774
    def cmd_rename_attribute(self, etype, oldname, newname, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   775
        """rename an existing attribute of the given entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   776
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   777
        `oldname` is a string giving the name of the existing attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   778
        `newname` is a string giving the name of the renamed attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   779
        """
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   780
        eschema = self.fs_schema.eschema(etype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   781
        attrtype = eschema.destination(newname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   782
        # have to commit this first step anyway to get the definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   783
        # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   784
        self.cmd_add_attribute(etype, newname, attrtype, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   785
        # skipp NULL values if the attribute is required
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   786
        rql = 'SET X %s VAL WHERE X is %s, X %s VAL' % (newname, etype, oldname)
4633
a85b4361fb22 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4580
diff changeset
   787
        card = eschema.rdef(newname).cardinality[0]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   788
        if card == '1':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   789
            rql += ', NOT X %s NULL' % oldname
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   790
        self.rqlexec(rql, ask_confirm=self.verbosity>=2)
3548
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   791
        # XXX if both attributes fulltext indexed, should skip fti rebuild
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   792
        # XXX if old attribute was fti indexed but not the new one old value
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   793
        # won't be removed from the index (this occurs on other kind of
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   794
        # fulltextindexed change...)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   795
        self.cmd_drop_attribute(etype, oldname, commit=commit)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   796
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   797
    def cmd_add_entity_type(self, etype, auto=True, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   798
        """register a new entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   799
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   800
        in auto mode, automatically register entity's relation where the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   801
        targeted type is known
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   802
        """
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   803
        instschema = self.repo.schema
7166
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7161
diff changeset
   804
        eschema = self.fs_schema.eschema(etype)
7558
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   805
        if etype in instschema and not (eschema.final and eschema.eid is None):
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   806
            print 'warning: %s already known, skip addition' % etype
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   807
            return
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   808
        confirm = self.verbosity >= 2
4043
39ae94e0c8b8 give group mapping where needed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4042
diff changeset
   809
        groupmap = self.group_mapping()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   810
        cstrtypemap = self.cstrtype_mapping()
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   811
        # register the entity into CWEType
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   812
        execute = self._cw.execute
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   813
        ss.execschemarql(execute, eschema, ss.eschema2rql(eschema, groupmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   814
        # add specializes relation if needed
5293
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   815
        specialized = eschema.specializes()
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   816
        if specialized:
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   817
            try:
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   818
                specialized.eid = instschema[specialized].eid
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   819
            except KeyError:
6323
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   820
                raise ExecutionError('trying to add entity type but parent type is '
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   821
                                     'not yet in the database schema')
5293
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   822
            self.rqlexecall(ss.eschemaspecialize2rql(eschema), ask_confirm=confirm)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   823
        # register entity's attributes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   824
        for rschema, attrschema in eschema.attribute_definitions():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   825
            # ignore those meta relations, they will be automatically added
2617
89c62b855f2e [R schema hooks] rename META_RELATIONS_TYPES into META_RTYPES, use it and other schema consts to avoid errors
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2616
diff changeset
   826
            if rschema.type in META_RTYPES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   827
                continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   828
            if not rschema.type in instschema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   829
                # need to add the relation type and to commit to get it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   830
                # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   831
                self.cmd_add_relation_type(rschema.type, False, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   832
            # register relation definition
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   833
            rdef = self._get_rdef(rschema, eschema, eschema.destination(rschema))
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   834
            ss.execschemarql(execute, rdef, ss.rdef2rql(rdef, cstrtypemap, groupmap),)
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   835
        # take care to newly introduced base class
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   836
        # XXX some part of this should probably be under the "if auto" block
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   837
        for spschema in eschema.specialized_by(recursive=False):
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   838
            try:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   839
                instspschema = instschema[spschema]
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   840
            except KeyError:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   841
                # specialized entity type not in schema, ignore
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   842
                continue
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   843
            if instspschema.specializes() != eschema:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   844
                self.rqlexec('SET D specializes P WHERE D eid %(d)s, P name %(pn)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   845
                             {'d': instspschema.eid, 'pn': eschema.type},
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   846
                             ask_confirm=confirm)
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   847
                for rschema, tschemas, role in spschema.relation_definitions(True):
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   848
                    for tschema in tschemas:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   849
                        if not tschema in instschema:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   850
                            continue
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   851
                        if role == 'subject':
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   852
                            subjschema = spschema
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   853
                            objschema = tschema
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3582
diff changeset
   854
                            if rschema.final and rschema in instspschema.subjrels:
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   855
                                # attribute already set, has_rdef would check if
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   856
                                # it's of the same type, we don't want this so
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   857
                                # simply skip here
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   858
                                continue
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   859
                        elif role == 'object':
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   860
                            subjschema = tschema
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   861
                            objschema = spschema
4633
a85b4361fb22 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4580
diff changeset
   862
                        if (rschema.rdef(subjschema, objschema).infered
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   863
                            or (instschema.has_relation(rschema) and
4633
a85b4361fb22 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4580
diff changeset
   864
                                (subjschema, objschema) in instschema[rschema].rdefs)):
4721
8f63691ccb7f pylint style fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   865
                            continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   866
                        self.cmd_add_relation_definition(
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   867
                            subjschema.type, rschema.type, objschema.type)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   868
        if auto:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   869
            # we have commit here to get relation types actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   870
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   871
            added = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   872
            for rschema in eschema.subject_relations():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   873
                # attribute relation have already been processed and
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   874
                # 'owned_by'/'created_by' will be automatically added
2617
89c62b855f2e [R schema hooks] rename META_RELATIONS_TYPES into META_RTYPES, use it and other schema consts to avoid errors
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2616
diff changeset
   875
                if rschema.final or rschema.type in META_RTYPES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   876
                    continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   877
                rtypeadded = rschema.type in instschema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   878
                for targetschema in rschema.objects(etype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   879
                    # ignore relations where the targeted type is not in the
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2275
diff changeset
   880
                    # current instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   881
                    targettype = targetschema.type
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   882
                    if not targettype in instschema and targettype != etype:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   883
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   884
                    if not rtypeadded:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   885
                        # need to add the relation type and to commit to get it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   886
                        # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   887
                        added.append(rschema.type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   888
                        self.cmd_add_relation_type(rschema.type, False, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   889
                        rtypeadded = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   890
                    # register relation definition
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4367
diff changeset
   891
                    # remember this two avoid adding twice non symmetric relation
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   892
                    # such as "Emailthread forked_from Emailthread"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   893
                    added.append((etype, rschema.type, targettype))
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   894
                    rdef = self._get_rdef(rschema, eschema, targetschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   895
                    ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   896
                                     ss.rdef2rql(rdef, cstrtypemap, groupmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   897
            for rschema in eschema.object_relations():
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   898
                if rschema.type in META_RTYPES:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   899
                    continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   900
                rtypeadded = rschema.type in instschema or rschema.type in added
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   901
                for targetschema in rschema.subjects(etype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   902
                    # ignore relations where the targeted type is not in the
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2275
diff changeset
   903
                    # current instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   904
                    targettype = targetschema.type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   905
                    # don't check targettype != etype since in this case the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   906
                    # relation has already been added as a subject relation
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   907
                    if not targettype in instschema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   908
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   909
                    if not rtypeadded:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   910
                        # need to add the relation type and to commit to get it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   911
                        # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   912
                        self.cmd_add_relation_type(rschema.type, False, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   913
                        rtypeadded = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   914
                    elif (targettype, rschema.type, etype) in added:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   915
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   916
                    # register relation definition
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   917
                    rdef = self._get_rdef(rschema, targetschema, eschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   918
                    ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   919
                                     ss.rdef2rql(rdef, cstrtypemap, groupmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   920
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   921
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   922
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   923
    def cmd_drop_entity_type(self, etype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   924
        """unregister an existing entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   925
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   926
        This will trigger deletion of necessary relation types and definitions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   927
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   928
        # XXX what if we delete an entity type which is specialized by other types
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   929
        # unregister the entity from CWEType
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   930
        self.rqlexec('DELETE CWEType X WHERE X name %(etype)s', {'etype': etype},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   931
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   932
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   933
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   934
6115
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   935
    def cmd_rename_entity_type(self, oldname, newname, attrs=None, commit=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   936
        """rename an existing entity type in the persistent schema
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   937
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   938
        `oldname` is a string giving the name of the existing entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   939
        `newname` is a string giving the name of the renamed entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   940
        """
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   941
        schema = self.repo.schema
9162
bf0a575d8da8 [migraction] rename_entity_type simply warn if old entity type isn't in the schema. Closes #3004069
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8807
diff changeset
   942
        if oldname not in schema:
bf0a575d8da8 [migraction] rename_entity_type simply warn if old entity type isn't in the schema. Closes #3004069
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8807
diff changeset
   943
            print 'warning: entity type %s is unknown, skip renaming' % oldname
bf0a575d8da8 [migraction] rename_entity_type simply warn if old entity type isn't in the schema. Closes #3004069
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8807
diff changeset
   944
            return
bf0a575d8da8 [migraction] rename_entity_type simply warn if old entity type isn't in the schema. Closes #3004069
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8807
diff changeset
   945
        # if merging two existing entity types
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   946
        if newname in schema:
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   947
            assert oldname in ETYPE_NAME_MAP, \
6115
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   948
                   '%s should be mapped to %s in ETYPE_NAME_MAP' % (oldname,
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   949
                                                                    newname)
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   950
            if attrs is None:
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   951
                attrs = ','.join(SQL_PREFIX + rschema.type
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   952
                                 for rschema in schema[newname].subject_relations()
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   953
                                 if (rschema.final or rschema.inlined)
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   954
                                 and not rschema in PURE_VIRTUAL_RTYPES)
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   955
            else:
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   956
                attrs += ('eid', 'creation_date', 'modification_date', 'cwuri')
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   957
                attrs = ','.join(SQL_PREFIX + attr for attr in attrs)
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   958
            self.sqlexec('INSERT INTO %s%s(%s) SELECT %s FROM %s%s' % (
5999
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   959
                SQL_PREFIX, newname, attrs, attrs, SQL_PREFIX, oldname),
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   960
                         ask_confirm=False)
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   961
            # old entity type has not been added to the schema, can't gather it
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   962
            new = schema.eschema(newname)
6117
f3fdfcbda54b cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6116
diff changeset
   963
            oldeid = self.rqlexec('CWEType ET WHERE ET name %(on)s',
f3fdfcbda54b cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6116
diff changeset
   964
                                  {'on': oldname}, ask_confirm=False)[0][0]
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   965
            # backport old type relations to new type
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   966
            # XXX workflows, other relations?
5999
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   967
            for r1, rr1 in [('from_entity', 'to_entity'),
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   968
                            ('to_entity', 'from_entity')]:
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   969
                self.rqlexec('SET X %(r1)s NET WHERE X %(r1)s OET, '
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   970
                             'NOT EXISTS(X2 %(r1)s NET, X relation_type XRT, '
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   971
                             'X2 relation_type XRT, X %(rr1)s XTE, X2 %(rr1)s XTE), '
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   972
                             'OET eid %%(o)s, NET eid %%(n)s' % locals(),
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   973
                             {'o': oldeid, 'n': new.eid}, ask_confirm=False)
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   974
            # backport is / is_instance_of relation to new type
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   975
            for rtype in ('is', 'is_instance_of'):
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   976
                self.sqlexec('UPDATE %s_relation SET eid_to=%s WHERE eid_to=%s'
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   977
                             % (rtype, new.eid, oldeid), ask_confirm=False)
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   978
            # delete relations using SQL to avoid relations content removal
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   979
            # triggered by schema synchronization hooks.
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   980
            session = self.session
6116
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   981
            for rdeftype in ('CWRelation', 'CWAttribute'):
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   982
                thispending = set( (eid for eid, in self.sqlexec(
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   983
                    'SELECT cw_eid FROM cw_%s WHERE cw_from_entity=%%(eid)s OR '
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   984
                    ' cw_to_entity=%%(eid)s' % rdeftype,
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   985
                    {'eid': oldeid}, ask_confirm=False)) )
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   986
                # we should add deleted eids into pending eids else we may
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   987
                # get some validation error on commit since integrity hooks
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   988
                # may think some required relation is missing... This also ensure
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   989
                # repository caches are properly cleanup
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   990
                hook.CleanupDeletedEidsCacheOp.get_instance(session).union(thispending)
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   991
                # and don't forget to remove record from system tables
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   992
                entities = [session.entity_from_eid(eid, rdeftype) for eid in thispending]
9451
c83a8ecb9bf5 [multi-sources-removal] Simplify repo.delete_info_multi arguments, uri is no more used
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9402
diff changeset
   993
                self.repo.system_source.delete_info_multi(session, entities)
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   994
                self.sqlexec('DELETE FROM cw_%s WHERE cw_from_entity=%%(eid)s OR '
6116
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   995
                             'cw_to_entity=%%(eid)s' % rdeftype,
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   996
                             {'eid': oldeid}, ask_confirm=False)
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   997
                # now we have to manually cleanup relations pointing to deleted
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   998
                # entities
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   999
                thiseids = ','.join(str(eid) for eid in thispending)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1000
                for rschema, ttypes, role in schema[rdeftype].relation_definitions():
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1001
                    if rschema.type in VIRTUAL_RTYPES:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1002
                        continue
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1003
                    sqls = []
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1004
                    if role == 'object':
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1005
                        if rschema.inlined:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1006
                            for eschema in ttypes:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1007
                                sqls.append('DELETE FROM cw_%s WHERE cw_%s IN(%%s)'
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1008
                                            % (eschema, rschema))
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1009
                        else:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1010
                            sqls.append('DELETE FROM %s_relation WHERE eid_to IN(%%s)'
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1011
                                        % rschema)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1012
                    elif not rschema.inlined:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1013
                        sqls.append('DELETE FROM %s_relation WHERE eid_from IN(%%s)'
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1014
                                    % rschema)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1015
                    for sql in sqls:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1016
                        self.sqlexec(sql % thiseids, ask_confirm=False)
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
  1017
            # remove the old type: use rql to propagate deletion
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
  1018
            self.rqlexec('DELETE CWEType ET WHERE ET name %(on)s', {'on': oldname},
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
  1019
                         ask_confirm=False)
9162
bf0a575d8da8 [migraction] rename_entity_type simply warn if old entity type isn't in the schema. Closes #3004069
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8807
diff changeset
  1020
        # elif simply renaming an entity type
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1021
        else:
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
  1022
            self.rqlexec('SET ET name %(newname)s WHERE ET is CWEType, ET name %(on)s',
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
  1023
                         {'newname' : unicode(newname), 'on' : oldname},
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1024
                         ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1025
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1026
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1027
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1028
    def cmd_add_relation_type(self, rtype, addrdef=True, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1029
        """register a new relation type named `rtype`, as described in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1030
        schema description file.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1031
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1032
        `addrdef` is a boolean value; when True, it will also add all relations
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1033
        of the type just added found in the schema definition file. Note that it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1034
        implies an intermediate "commit" which commits the relation type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1035
        creation (but not the relation definitions themselves, for which
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1036
        committing depends on the `commit` argument value).
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1037
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1038
        """
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1039
        reposchema = self.repo.schema
8209
67660d5f1916 [migration] `add_relation_type` warns and add missing definitions on already existing rtypes (closes #2121979)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8049
diff changeset
  1040
        rschema = self.fs_schema.rschema(rtype)
67660d5f1916 [migration] `add_relation_type` warns and add missing definitions on already existing rtypes (closes #2121979)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8049
diff changeset
  1041
        execute = self._cw.execute
7558
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
  1042
        if rtype in reposchema:
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1043
            print 'warning: relation type %s is already known, skip addition' % (
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1044
                rtype)
8209
67660d5f1916 [migration] `add_relation_type` warns and add missing definitions on already existing rtypes (closes #2121979)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8049
diff changeset
  1045
        else:
67660d5f1916 [migration] `add_relation_type` warns and add missing definitions on already existing rtypes (closes #2121979)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8049
diff changeset
  1046
            # register the relation into CWRType and insert necessary relation
67660d5f1916 [migration] `add_relation_type` warns and add missing definitions on already existing rtypes (closes #2121979)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8049
diff changeset
  1047
            # definitions
67660d5f1916 [migration] `add_relation_type` warns and add missing definitions on already existing rtypes (closes #2121979)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8049
diff changeset
  1048
            ss.execschemarql(execute, rschema, ss.rschema2rql(rschema, addrdef=False))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1049
        if addrdef:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1050
            self.commit()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1051
            gmap = self.group_mapping()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1052
            cmap = self.cstrtype_mapping()
6073
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1053
            done = set()
8285
32098d98bf2f [migration] fix add_relation_type in case it already exists, broken by 67660d5f1916
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8278
diff changeset
  1054
            for subj, obj in rschema.rdefs:
32098d98bf2f [migration] fix add_relation_type in case it already exists, broken by 67660d5f1916
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8278
diff changeset
  1055
                if not (reposchema.has_entity(subj)
32098d98bf2f [migration] fix add_relation_type in case it already exists, broken by 67660d5f1916
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8278
diff changeset
  1056
                        and reposchema.has_entity(obj)):
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1057
                    continue
6073
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1058
                # symmetric relations appears twice
8285
32098d98bf2f [migration] fix add_relation_type in case it already exists, broken by 67660d5f1916
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8278
diff changeset
  1059
                if (subj, obj) in done:
6073
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1060
                    continue
8285
32098d98bf2f [migration] fix add_relation_type in case it already exists, broken by 67660d5f1916
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8278
diff changeset
  1061
                done.add( (subj, obj) )
32098d98bf2f [migration] fix add_relation_type in case it already exists, broken by 67660d5f1916
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8278
diff changeset
  1062
                self.cmd_add_relation_definition(subj, rtype, obj)
2700
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1063
            if rtype in META_RTYPES:
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1064
                # if the relation is in META_RTYPES, ensure we're adding it for
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1065
                # all entity types *in the persistent schema*, not only those in
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1066
                # the fs schema
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1067
                for etype in self.repo.schema.entities():
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1068
                    if not etype in self.fs_schema:
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1069
                        # get sample object type and rproperties
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1070
                        objtypes = rschema.objects()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1071
                        assert len(objtypes) == 1, objtypes
2700
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1072
                        objtype = objtypes[0]
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1073
                        rdef = copy(rschema.rdef(rschema.subjects(objtype)[0], objtype))
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1074
                        rdef.subject = etype
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1075
                        rdef.rtype = self.repo.schema.rschema(rschema)
8616
820cb9cee702 [migration] fix crash when adding a new meta relation type. Closes #2532483
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8530
diff changeset
  1076
                        rdef.object = self.repo.schema.eschema(objtype)
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1077
                        ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1078
                                         ss.rdef2rql(rdef, cmap, gmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1079
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1080
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1081
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1082
    def cmd_drop_relation_type(self, rtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1083
        """unregister an existing relation type"""
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1084
        # unregister the relation from CWRType
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1085
        self.rqlexec('DELETE CWRType X WHERE X name %r' % rtype,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1086
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1087
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1088
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1089
8467
ad75430a2dc8 [migraction] rename rename_relation method to rename_relation_type for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8307
diff changeset
  1090
    def cmd_rename_relation_type(self, oldname, newname, commit=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1091
        """rename an existing relation
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1092
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1093
        `oldname` is a string giving the name of the existing relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1094
        `newname` is a string giving the name of the renamed relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1095
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1096
        self.cmd_add_relation_type(newname, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1097
        self.rqlexec('SET X %s Y WHERE X %s Y' % (newname, oldname),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1098
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1099
        self.cmd_drop_relation_type(oldname, commit=commit)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1100
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1101
    def cmd_add_relation_definition(self, subjtype, rtype, objtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1102
        """register a new relation definition, from its definition found in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1103
        schema definition file
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1104
        """
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
  1105
        rschema = self.fs_schema.rschema(rtype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1106
        if not rtype in self.repo.schema:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1107
            self.cmd_add_relation_type(rtype, addrdef=False, commit=True)
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1108
        if (subjtype, objtype) in self.repo.schema.rschema(rtype).rdefs:
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1109
            print 'warning: relation %s %s %s is already known, skip addition' % (
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1110
                subjtype, rtype, objtype)
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1111
            return
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1112
        rdef = self._get_rdef(rschema, subjtype, objtype)
8285
32098d98bf2f [migration] fix add_relation_type in case it already exists, broken by 67660d5f1916
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8278
diff changeset
  1113
        ss.execschemarql(self._cw.execute, rdef,
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1114
                         ss.rdef2rql(rdef, self.cstrtype_mapping(),
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1115
                                     self.group_mapping()))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1116
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1117
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1118
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1119
    def _get_rdef(self, rschema, subjtype, objtype):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1120
        return self._set_rdef_eid(rschema.rdefs[(subjtype, objtype)])
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1121
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1122
    def _set_rdef_eid(self, rdef):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1123
        for attr in ('rtype', 'subject', 'object'):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1124
            schemaobj = getattr(rdef, attr)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1125
            if getattr(schemaobj, 'eid', None) is None:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1126
                schemaobj.eid =  self.repo.schema[schemaobj].eid
8278
3e8b94d921e0 [migration] add schema obj into assertion failure message so we get a chance to understand the pb
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8232
diff changeset
  1127
                assert schemaobj.eid is not None, schemaobj
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1128
        return rdef
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1129
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1130
    def cmd_drop_relation_definition(self, subjtype, rtype, objtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1131
        """unregister an existing relation definition"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1132
        rschema = self.repo.schema.rschema(rtype)
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1133
        # unregister the definition from CWAttribute or CWRelation
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3582
diff changeset
  1134
        if rschema.final:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1135
            etype = 'CWAttribute'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1136
        else:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1137
            etype = 'CWRelation'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1138
        rql = ('DELETE %s X WHERE X from_entity FE, FE name "%s",'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1139
               'X relation_type RT, RT name "%s", X to_entity TE, TE name "%s"')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1140
        self.rqlexec(rql % (etype, subjtype, rtype, objtype),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1141
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1142
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1143
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1144
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1145
    def cmd_sync_schema_props_perms(self, ertype=None, syncperms=True,
1409
f4dee84a618f sql attributes bugfix
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 1399
diff changeset
  1146
                                    syncprops=True, syncrdefs=True, commit=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1147
        """synchronize the persistent schema against the current definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1148
        schema.
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1149
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1150
        `ertype` can be :
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1151
        - None, in that case everything will be synced ;
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1152
        - a string, it should be an entity type or
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1153
          a relation type. In that case, only the corresponding
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1154
          entities / relations will be synced ;
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1155
        - an rdef object to synchronize only this specific relation definition
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1156
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1157
        It will synch common stuff between the definition schema and the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1158
        actual persistent schema, it won't add/remove any entity or relation.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1159
        """
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1160
        assert syncperms or syncprops, 'nothing to do'
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1161
        if ertype is not None:
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1162
            if isinstance(ertype, RelationDefinitionSchema):
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1163
                ertype = ertype.as_triple()
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1164
            if isinstance(ertype, (tuple, list)):
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1165
                assert len(ertype) == 3, 'not a relation definition'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
  1166
                self._synchronize_rdef_schema(ertype[0], ertype[1], ertype[2],
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1167
                                              syncperms=syncperms,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1168
                                              syncprops=syncprops)
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1169
            else:
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1170
                erschema = self.repo.schema[ertype]
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1171
                if isinstance(erschema, CubicWebRelationSchema):
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1172
                    self._synchronize_rschema(erschema, syncrdefs=syncrdefs,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1173
                                              syncperms=syncperms,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1174
                                              syncprops=syncprops)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1175
                else:
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1176
                    self._synchronize_eschema(erschema, syncrdefs=syncrdefs,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1177
                                              syncperms=syncperms,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1178
                                              syncprops=syncprops)
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1179
        else:
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1180
            for etype in self.repo.schema.entities():
7505
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1181
                if etype.eid is None:
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1182
                     # not yet added final etype (thing to BigInt defined in
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1183
                     # yams though 3.13 migration not done yet)
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1184
                    continue
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1185
                self._synchronize_eschema(etype, syncrdefs=syncrdefs,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1186
                                          syncprops=syncprops, syncperms=syncperms)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1187
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1188
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1189
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1190
    def cmd_change_relation_props(self, subjtype, rtype, objtype,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1191
                                  commit=True, **kwargs):
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1192
        """change some properties of a relation definition
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1193
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1194
        you usually want to use sync_schema_props_perms instead.
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1195
        """
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1196
        assert kwargs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1197
        restriction = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1198
        if subjtype and subjtype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1199
            restriction.append('X from_entity FE, FE name "%s"' % subjtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1200
        if objtype and objtype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1201
            restriction.append('X to_entity TE, TE name "%s"' % objtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1202
        if rtype and rtype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1203
            restriction.append('X relation_type RT, RT name "%s"' % rtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1204
        assert restriction
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1205
        values = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1206
        for k, v in kwargs.items():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1207
            values.append('X %s %%(%s)s' % (k, k))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1208
            if isinstance(v, str):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1209
                kwargs[k] = unicode(v)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1210
        rql = 'SET %s WHERE %s' % (','.join(values), ','.join(restriction))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1211
        self.rqlexec(rql, kwargs, ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1212
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1213
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1214
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1215
    def cmd_set_size_constraint(self, etype, rtype, size, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1216
        """set change size constraint of a string attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1217
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1218
        if size is None any size constraint will be removed.
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1219
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1220
        you usually want to use sync_schema_props_perms instead.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1221
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1222
        oldvalue = None
4695
4aaf87e7f79e 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4650
diff changeset
  1223
        for constr in self.repo.schema.eschema(etype).rdef(rtype).constraints:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1224
            if isinstance(constr, SizeConstraint):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1225
                oldvalue = constr.max
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1226
        if oldvalue == size:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1227
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1228
        if oldvalue is None and not size is None:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1229
            ceid = self.rqlexec('INSERT CWConstraint C: C value %(v)s, C cstrtype CT '
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1230
                                'WHERE CT name "SizeConstraint"',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1231
                                {'v': SizeConstraint(size).serialize()},
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1232
                                ask_confirm=self.verbosity>=2)[0][0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1233
            self.rqlexec('SET X constrained_by C WHERE X from_entity S, X relation_type R, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1234
                         'S name "%s", R name "%s", C eid %s' % (etype, rtype, ceid),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1235
                         ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1236
        elif not oldvalue is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1237
            if not size is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1238
                self.rqlexec('SET C value %%(v)s WHERE X from_entity S, X relation_type R,'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1239
                             'X constrained_by C, C cstrtype CT, CT name "SizeConstraint",'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1240
                             'S name "%s", R name "%s"' % (etype, rtype),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1241
                             {'v': unicode(SizeConstraint(size).serialize())},
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1242
                             ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1243
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1244
                self.rqlexec('DELETE X constrained_by C WHERE X from_entity S, X relation_type R,'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1245
                             'X constrained_by C, C cstrtype CT, CT name "SizeConstraint",'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1246
                             'S name "%s", R name "%s"' % (etype, rtype),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1247
                             ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1248
                # cleanup unused constraints
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1249
                self.rqlexec('DELETE CWConstraint C WHERE NOT X constrained_by C')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1250
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1251
            self.commit()
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1252
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1253
    # Workflows handling ######################################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1254
6805
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1255
    def cmd_make_workflowable(self, etype):
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1256
        """add workflow relations to an entity type to make it workflowable"""
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1257
        self.cmd_add_relation_definition(etype, 'in_state', 'State')
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1258
        self.cmd_add_relation_definition(etype, 'custom_workflow', 'Workflow')
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1259
        self.cmd_add_relation_definition('TrInfo', 'wf_info_for', etype)
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1260
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1261
    def cmd_add_workflow(self, name, wfof, default=True, commit=False,
6816
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1262
                         ensure_workflowable=True, **kwargs):
3582
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1263
        """
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1264
        create a new workflow and links it to entity types
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1265
         :type name: unicode
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1266
         :param name: name of the workflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1267
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1268
         :type wfof: string or list/tuple of strings
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1269
         :param wfof: entity type(s) having this workflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1270
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1271
         :type default: bool
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1272
         :param default: tells wether this is the default workflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1273
                   for the specified entity type(s); set it to false in
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1274
                   the case of a subworkflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1275
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1276
         :rtype: `Workflow`
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1277
        """
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1278
        wf = self.cmd_create_entity('Workflow', name=unicode(name),
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1279
                                    **kwargs)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1280
        if not isinstance(wfof, (list, tuple)):
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1281
            wfof = (wfof,)
6805
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1282
        def _missing_wf_rel(etype):
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1283
            return 'missing workflow relations, see make_workflowable(%s)' % etype
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1284
        for etype in wfof:
6805
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1285
            eschema = self.repo.schema[etype]
6816
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1286
            if ensure_workflowable:
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1287
                assert 'in_state' in eschema.subjrels, _missing_wf_rel(etype)
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1288
                assert 'custom_workflow' in eschema.subjrels, _missing_wf_rel(etype)
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1289
                assert 'wf_info_for' in eschema.objrels, _missing_wf_rel(etype)
2956
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1290
            rset = self.rqlexec(
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1291
                'SET X workflow_of ET WHERE X eid %(x)s, ET name %(et)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1292
                {'x': wf.eid, 'et': etype}, ask_confirm=False)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1293
            assert rset, 'unexistant entity type %s' % etype
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1294
            if default:
2956
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1295
                self.rqlexec(
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1296
                    'SET ET default_workflow X WHERE X eid %(x)s, ET name %(et)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1297
                    {'x': wf.eid, 'et': etype}, ask_confirm=False)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1298
        if commit:
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1299
            self.commit()
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1300
        return wf
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1301
7295
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1302
    def cmd_get_workflow_for(self, etype):
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1303
        """return default workflow for the given entity type"""
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1304
        rset = self.rqlexec('Workflow X WHERE ET default_workflow X, ET name %(et)s',
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1305
                            {'et': etype})
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1306
        return rset.get_entity(0, 0)
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1307
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1308
    # CWProperty handling ######################################################
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1309
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1310
    def cmd_property_value(self, pkey):
7111
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1311
        """retreive the site-wide persistent property value for the given key.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1312
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1313
        To get a user specific property value, use appropriate method on CWUser
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1314
        instance.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1315
        """
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1316
        rset = self.rqlexec(
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1317
            'Any V WHERE X is CWProperty, X pkey %(k)s, X value V, NOT X for_user U',
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1318
            {'k': pkey}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1319
        return rset[0][0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1320
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1321
    def cmd_set_property(self, pkey, value):
7111
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1322
        """set the site-wide persistent property value for the given key to the
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1323
        given value.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1324
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1325
        To set a user specific property value, use appropriate method on CWUser
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1326
        instance.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1327
        """
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1328
        value = unicode(value)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1329
        try:
7111
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1330
            prop = self.rqlexec(
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1331
                'CWProperty X WHERE X pkey %(k)s, NOT X for_user U',
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1332
                {'k': pkey}, ask_confirm=False).get_entity(0, 0)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
  1333
        except Exception:
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1334
            self.cmd_create_entity('CWProperty', pkey=unicode(pkey), value=value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1335
        else:
8483
4ba11607d84a [entity api] unify set_attributes / set_relations into a cw_set method. Closes #2423719
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8467
diff changeset
  1336
            prop.cw_set(value=value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1337
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1338
    # other data migration commands ###########################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1339
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1340
    @property
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1341
    def _cw(self):
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1342
        session = self.session
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1343
        if session is not None:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1344
            session.set_cnxset()
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1345
            return session
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1346
        return self.cnx.request()
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1347
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1348
    def cmd_storage_changed(self, etype, attribute):
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1349
        """migrate entities to a custom storage. The new storage is expected to
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1350
        be set, it will be temporarily removed for the migration.
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1351
        """
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1352
        from logilab.common.shellutils import ProgressBar
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1353
        source = self.repo.system_source
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1354
        storage = source.storage(etype, attribute)
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1355
        source.unset_storage(etype, attribute)
5552
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5551
diff changeset
  1356
        rset = self.rqlexec('Any X WHERE X is %s' % etype, ask_confirm=False)
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1357
        pb = ProgressBar(len(rset))
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1358
        for entity in rset.entities():
5551
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1359
            # fill cache. Do not fetch that attribute using the global rql query
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1360
            # since we may exhaust memory doing that....
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1361
            getattr(entity, attribute)
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1362
            storage.migrate_entity(entity, attribute)
5551
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1363
            # remove from entity cache to avoid memory exhaustion
8231
1bb43e31032d [bfss, migration] fix storage migration (closes #2189206)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8047
diff changeset
  1364
            del entity.cw_attr_cache[attribute]
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1365
            pb.update()
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1366
        print
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1367
        source.set_storage(etype, attribute, storage)
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1368
5132
260d73ad4f24 [cleaning] simpler implementation of cmd_create_entity
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5043
diff changeset
  1369
    def cmd_create_entity(self, etype, commit=False, **kwargs):
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1370
        """add a new entity of the given type"""
3708
95e8c3a9698a update prototype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3707
diff changeset
  1371
        entity = self._cw.create_entity(etype, **kwargs)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1372
        if commit:
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1373
            self.commit()
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1374
        return entity
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1375
7116
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1376
    def cmd_find_entities(self, etype, **kwargs):
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1377
        """find entities of the given type and attribute values"""
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1378
        return self._cw.find_entities(etype, **kwargs)
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1379
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1380
    def cmd_find_one_entity(self, etype, **kwargs):
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1381
        """find one entity of the given type and attribute values.
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1382
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1383
        raise :exc:`cubicweb.req.FindEntityError` if can not return one and only
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1384
        one entity.
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1385
        """
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1386
        return self._cw.find_one_entity(etype, **kwargs)
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1387
5928
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1388
    def cmd_update_etype_fti_weight(self, etype, weight):
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1389
        if self.repo.system_source.dbdriver == 'postgres':
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1390
            self.sqlexec('UPDATE appears SET weight=%(weight)s '
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1391
                         'FROM entities as X '
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1392
                         'WHERE X.eid=appears.uid AND X.type=%(type)s',
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1393
                         {'type': etype, 'weight': weight}, ask_confirm=False)
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1394
5850
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1395
    def cmd_reindex_entities(self, etypes=None):
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1396
        """force reindexaction of entities of the given types or of all
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1397
        indexable entity types
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1398
        """
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1399
        from cubicweb.server.checkintegrity import reindex_entities
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1400
        reindex_entities(self.repo.schema, self.session, etypes=etypes)
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1401
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1402
    @contextmanager
7409
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1403
    def cmd_dropped_constraints(self, etype, attrname, cstrtype=None,
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1404
                                droprequired=False):
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1405
        """context manager to drop constraints temporarily on fs_schema
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1406
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1407
        `cstrtype` should be a constraint class (or a tuple of classes)
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1408
        and will be passed to isinstance directly
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1409
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1410
        For instance::
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1411
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1412
            >>> with dropped_constraints('MyType', 'myattr',
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1413
            ...                          UniqueConstraint, droprequired=True):
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1414
            ...     add_attribute('MyType', 'myattr')
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1415
            ...     # + instructions to fill MyType.myattr column
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1416
            ...
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1417
            >>>
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1418
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1419
        """
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1420
        rdef = self.fs_schema.eschema(etype).rdef(attrname)
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1421
        original_constraints = rdef.constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1422
        # remove constraints
7409
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1423
        if cstrtype:
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1424
            rdef.constraints = [cstr for cstr in original_constraints
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1425
                                if not (cstrtype and isinstance(cstr, cstrtype))]
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1426
        if droprequired:
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1427
            original_cardinality = rdef.cardinality
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1428
            rdef.cardinality = '?' + rdef.cardinality[1]
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1429
        yield
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1430
        # restore original constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1431
        rdef.constraints = original_constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1432
        if droprequired:
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1433
            rdef.cardinality = original_cardinality
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1434
        # update repository schema
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1435
        self.cmd_sync_schema_props_perms(rdef, syncperms=False)
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1436
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1437
    def sqlexec(self, sql, args=None, ask_confirm=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1438
        """execute the given sql if confirmed
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1439
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1440
        should only be used for low level stuff undoable with existing higher
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1441
        level actions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1442
        """
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1443
        if not ask_confirm or self.confirm('Execute sql: %s ?' % sql):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1444
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1445
                cu = self.session.system_sql(sql, args)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
  1446
            except Exception:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1447
                ex = sys.exc_info()[1]
6187
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6186
diff changeset
  1448
                if self.confirm('Error: %s\nabort?' % ex, pdb=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1449
                    raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1450
                return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1451
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1452
                return cu.fetchall()
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
  1453
            except Exception:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1454
                # no result to fetch
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1455
                return
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1456
9255
46f41c3e1443 remove 3.8 bw compat
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9241
diff changeset
  1457
    def rqlexec(self, rql, kwargs=None, build_descr=True,
6186
8ada7df95877 [migration] set ask_confirm=False by default on rqlexec
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6179
diff changeset
  1458
                ask_confirm=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1459
        """rql action"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1460
        if not isinstance(rql, (tuple, list)):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1461
            rql = ( (rql, kwargs), )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1462
        res = None
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1463
        execute = self._cw.execute
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1464
        for rql, kwargs in rql:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1465
            if kwargs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1466
                msg = '%s (%s)' % (rql, kwargs)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1467
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1468
                msg = rql
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1469
            if not ask_confirm or self.confirm('Execute rql: %s ?' % msg):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1470
                try:
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1471
                    res = execute(rql, kwargs, build_descr=build_descr)
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8694
diff changeset
  1472
                except Exception as ex:
6187
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6186
diff changeset
  1473
                    if self.confirm('Error: %s\nabort?' % ex, pdb=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1474
                        raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1475
        return res
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1476
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1477
    def rqliter(self, rql, kwargs=None, ask_confirm=True):
6686
070e3b23160d [migration] fix rqliter when some query arguments are given
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6582
diff changeset
  1478
        return ForRqlIterator(self, rql, kwargs, ask_confirm)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1479
8181
166ad5458ed8 [migractions] fix comment to avoid ambiguity. Closes #2015260
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8049
diff changeset
  1480
    # low-level commands to repair broken system database ######################
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1481
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1482
    def cmd_change_attribute_type(self, etype, attr, newtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1483
        """low level method to change the type of an entity attribute. This is
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1484
        a quick hack which has some drawback:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1485
        * only works when the old type can be changed to the new type by the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1486
          underlying rdbms (eg using ALTER TABLE)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1487
        * the actual schema won't be updated until next startup
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1488
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1489
        rschema = self.repo.schema.rschema(attr)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1490
        oldtype = rschema.objects(etype)[0]
7928
c65fd337fd1d [migration] update old yams api in change_attribute_type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7915
diff changeset
  1491
        rdefeid = rschema.rdef(etype, oldtype).eid
7948
deb596dc343a [migraction] fix cmd_change_attribute_type command (closes #2015073)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7928
diff changeset
  1492
        allownull = rschema.rdef(etype, oldtype).cardinality[0] != '1'
7410
1dbf9a83bb49 [migractions] fix change_attribute_type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7409
diff changeset
  1493
        sql = ("UPDATE cw_CWAttribute "
1dbf9a83bb49 [migractions] fix change_attribute_type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7409
diff changeset
  1494
               "SET cw_to_entity=(SELECT cw_eid FROM cw_CWEType WHERE cw_name='%s')"
1dbf9a83bb49 [migractions] fix change_attribute_type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7409
diff changeset
  1495
               "WHERE cw_eid=%s") % (newtype, rdefeid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1496
        self.sqlexec(sql, ask_confirm=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1497
        dbhelper = self.repo.system_source.dbhelper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1498
        sqltype = dbhelper.TYPE_MAPPING[newtype]
9463
d62e13eba033 [multi-sources-removal] Simplify ConnectionsSet internal structures and public methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9460
diff changeset
  1499
        cursor = self.session.cnxset.cu
7948
deb596dc343a [migraction] fix cmd_change_attribute_type command (closes #2015073)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7928
diff changeset
  1500
        dbhelper.change_col_type(cursor, 'cw_%s'  % etype, 'cw_%s' % attr, sqltype, allownull)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1501
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1502
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1503
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1504
    def cmd_add_entity_type_table(self, etype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1505
        """low level method to create the sql table for an existing entity.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1506
        This may be useful on accidental desync between the repository schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1507
        and a sql database
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1508
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1509
        dbhelper = self.repo.system_source.dbhelper
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
  1510
        tablesql = eschema2sql(dbhelper, self.repo.schema.eschema(etype),
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
  1511
                               prefix=SQL_PREFIX)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1512
        for sql in tablesql.split(';'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1513
            if sql.strip():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1514
                self.sqlexec(sql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1515
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1516
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1517
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1518
    def cmd_add_relation_type_table(self, rtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1519
        """low level method to create the sql table for an existing relation.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1520
        This may be useful on accidental desync between the repository schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1521
        and a sql database
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1522
        """
7879
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
  1523
        tablesql = rschema2sql(self.repo.schema.rschema(rtype))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1524
        for sql in tablesql.split(';'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1525
            if sql.strip():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1526
                self.sqlexec(sql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1527
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1528
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1529
8530
2bceea9dee95 [migration] fix backward compat for rename_relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8467
diff changeset
  1530
    @deprecated("[3.15] use rename_relation_type(oldname, newname)")
8467
ad75430a2dc8 [migraction] rename rename_relation method to rename_relation_type for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8307
diff changeset
  1531
    def cmd_rename_relation(self, oldname, newname, commit=True):
8530
2bceea9dee95 [migration] fix backward compat for rename_relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8467
diff changeset
  1532
        self.cmd_rename_relation_type(oldname, newname, commit)
8467
ad75430a2dc8 [migraction] rename rename_relation method to rename_relation_type for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8307
diff changeset
  1533
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1534
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1535
class ForRqlIterator:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1536
    """specific rql iterator to make the loop skipable"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1537
    def __init__(self, helper, rql, kwargs, ask_confirm):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1538
        self._h = helper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1539
        self.rql = rql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1540
        self.kwargs = kwargs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1541
        self.ask_confirm = ask_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1542
        self._rsetit = None
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1543
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1544
    def __iter__(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1545
        return self
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1546
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1547
    def _get_rset(self):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1548
        rql, kwargs = self.rql, self.kwargs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1549
        if kwargs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1550
            msg = '%s (%s)' % (rql, kwargs)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1551
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1552
            msg = rql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1553
        if self.ask_confirm:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1554
            if not self._h.confirm('Execute rql: %s ?' % msg):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1555
                raise StopIteration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1556
        try:
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1557
            return self._h._cw.execute(rql, kwargs)
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8694
diff changeset
  1558
        except Exception as ex:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1559
            if self._h.confirm('Error: %s\nabort?' % ex):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1560
                raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1561
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1562
                raise StopIteration
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1563
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1564
    def next(self):
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1565
        if self._rsetit is not None:
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1566
            return self._rsetit.next()
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1567
        rset = self._get_rset()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1568
        self._rsetit = iter(rset)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1569
        return self._rsetit.next()
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1570
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1571
    def entities(self):
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1572
        try:
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1573
            rset = self._get_rset()
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1574
        except StopIteration:
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1575
            return []
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1576
        return rset.entities()