server/migractions.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Fri, 07 Oct 2011 17:29:14 +0200
branchstable
changeset 7928 c65fd337fd1d
parent 7915 a7f3245e1728
child 7931 60068dc83457
child 7948 deb596dc343a
permissions -rw-r--r--
[migration] update old yams api in change_attribute_type
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
7116
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
     1
# copyright 2003-2011 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
"""a class implementing basic actions used in migration scripts.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
The following schema actions are supported for now:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
* add/drop/rename attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
* add/drop entity/relation type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
* rename entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
The following data actions are supported for now:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
* add an entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
* execute raw RQL queries
5850
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
    28
"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    29
4925
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
    30
from __future__ import with_statement
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
    31
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    32
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    33
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    34
import sys
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    35
import os
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    36
import tarfile
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    37
import tempfile
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    38
import shutil
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    39
import os.path as osp
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 972
diff changeset
    40
from datetime import datetime
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
    41
from glob import glob
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
    42
from copy import copy
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
    43
from warnings import warn
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
    44
from contextlib import contextmanager
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    45
2613
5e19c2bb370e R [all] logilab.common 0.44 provides only deprecated
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2493
diff changeset
    46
from logilab.common.deprecation import deprecated
2107
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    47
from logilab.common.decorators import cached, clear_cache
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    48
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    49
from yams.constraints import SizeConstraint
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
from yams.schema2sql import eschema2sql, rschema2sql
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
    51
from yams.schema import RelationDefinitionSchema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
    53
from cubicweb import CW_SOFTWARE_ROOT, AuthenticationError, ExecutionError
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    54
from cubicweb.selectors import is_instance
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
    55
from cubicweb.schema import (ETYPE_NAME_MAP, META_RTYPES, VIRTUAL_RTYPES,
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
    56
                             PURE_VIRTUAL_RTYPES,
2926
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
    57
                             CubicWebRelationSchema, order_eschemas)
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    58
from cubicweb.cwvreg import CW_EVENT_MANAGER
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    59
from cubicweb.dbapi import get_repository, repo_connect
4021
280c910c8710 move i18n / migration modules from cw.common to cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4017
diff changeset
    60
from cubicweb.migration import MigrationHelper, yes
4925
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
    61
from cubicweb.server.session import hooks_control
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
    62
from cubicweb.server import hook
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
try:
2107
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    64
    from cubicweb.server import SOURCE_TYPES, schemaserial as ss
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
    65
    from cubicweb.server.utils import manager_userpasswd
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
    66
    from cubicweb.server.sqlutils import sqlexec, SQL_PREFIX
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    67
except ImportError: # LAX
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    68
    pass
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    69
6571
71b446d1cf93 [dependancy] do not use testlib from outside test: recent lgc versions recommends unittest2 while lgc.testlib needs it, and we don't want it to be necessary to run cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6323
diff changeset
    70
71b446d1cf93 [dependancy] do not use testlib from outside test: recent lgc versions recommends unittest2 while lgc.testlib needs it, and we don't want it to be necessary to run cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6323
diff changeset
    71
def mock_object(**params):
71b446d1cf93 [dependancy] do not use testlib from outside test: recent lgc versions recommends unittest2 while lgc.testlib needs it, and we don't want it to be necessary to run cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6323
diff changeset
    72
    return type('Mock', (), params)()
71b446d1cf93 [dependancy] do not use testlib from outside test: recent lgc versions recommends unittest2 while lgc.testlib needs it, and we don't want it to be necessary to run cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6323
diff changeset
    73
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    74
class ClearGroupMap(hook.Hook):
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    75
    __regid__ = 'cw.migration.clear_group_mapping'
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    76
    __select__ = hook.Hook.__select__ & is_instance('CWGroup')
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    77
    events = ('after_add_entity', 'after_update_entity',)
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    78
    def __call__(self):
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    79
        clear_cache(self.mih, 'group_mapping')
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    80
        self.mih._synchronized.clear()
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
    81
6205
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    82
    @classmethod
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    83
    def mih_register(cls, repo):
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    84
        # may be already registered in tests (e.g. unittest_migractions at
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    85
        # least)
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    86
        if not cls.__regid__ in repo.vreg['after_add_entity_hooks']:
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    87
            repo.vreg.register(ClearGroupMap)
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    88
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    89
class ServerMigrationHelper(MigrationHelper):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
    """specific migration helper for server side  migration scripts,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
    providind actions related to schema/data migration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    94
    def __init__(self, config, schema, interactive=True,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
                 repo=None, cnx=None, verbosity=1, connect=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
        MigrationHelper.__init__(self, config, interactive, verbosity)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    97
        if not interactive:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    98
            assert cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    99
            assert repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   100
        if cnx is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   101
            assert repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   102
            self._cnx = cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   103
            self.repo = repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   104
        elif connect:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   105
            self.repo_connect()
5019
72734c210836 [c-c] new server_maintenance hook, called on c-c shell / upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   106
        # no config on shell to a remote instance
5034
4781870e97d9 [maintainance] don't crash if we've no in-memory repository
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5019
diff changeset
   107
        if config is not None and (cnx or connect):
6203
d3dea5f84404 [test] during some tests (unittest_migrations), we should not register ClearGroupMap which is actually already there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6200
diff changeset
   108
            repo = self.repo
5019
72734c210836 [c-c] new server_maintenance hook, called on c-c shell / upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   109
            self.session.data['rebuild-infered'] = False
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   110
            # register a hook to clear our group_mapping cache and the
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   111
            # self._synchronized set when some group is added or updated
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   112
            ClearGroupMap.mih = self
6205
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
   113
            ClearGroupMap.mih_register(repo)
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
   114
            CW_EVENT_MANAGER.bind('after-registry-reload',
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
   115
                                  ClearGroupMap.mih_register, repo)
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   116
            # notify we're starting maintenance (called instead of server_start
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   117
            # which is called on regular start
6203
d3dea5f84404 [test] during some tests (unittest_migrations), we should not register ClearGroupMap which is actually already there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6200
diff changeset
   118
            repo.hm.call_hooks('server_maintenance', repo=repo)
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5034
diff changeset
   119
        if not schema and not getattr(config, 'quick_start', False):
7782
40a49f4350a5 backout 7780:a1d5365fefc1
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7780
diff changeset
   120
            schema = config.load_schema(expand_cubes=True)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   121
        self.fs_schema = schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   122
        self._synchronized = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   123
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   124
    # overriden from base MigrationHelper ######################################
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   125
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   126
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   127
    def repo_connect(self):
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   128
        self.repo = get_repository(method='inmemory', config=self.config)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
        return self.repo
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   130
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   131
    def cube_upgraded(self, cube, version):
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   132
        self.cmd_set_property('system.version.%s' % cube.lower(),
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   133
                              unicode(version))
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   134
        self.commit()
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   135
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
    def shutdown(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
        if self.repo is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   138
            self.repo.shutdown()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   139
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   140
    def migrate(self, vcconf, toupgrade, options):
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   141
        if not options.fs_only:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   142
            if options.backup_db is None:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   143
                self.backup_database()
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   144
            elif options.backup_db:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   145
                self.backup_database(askconfirm=False)
4925
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
   146
        # disable notification during migration
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
   147
        with hooks_control(self.session, self.session.HOOKS_ALLOW_ALL, 'notification'):
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
   148
            super(ServerMigrationHelper, self).migrate(vcconf, toupgrade, options)
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   149
3715
e3ccadb126d7 [shell] make process_script available throuhg c-c shell / migration script context
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3708
diff changeset
   150
    def cmd_process_script(self, migrscript, funcname=None, *args, **kwargs):
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   151
        try:
6042
df9cafb8062c [migration] commit only sql script migration
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6035
diff changeset
   152
            return super(ServerMigrationHelper, self).cmd_process_script(
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   153
                  migrscript, funcname, *args, **kwargs)
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   154
        except ExecutionError, err:
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   155
            print >> sys.stderr, "-> %s" % err
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   156
        except BaseException:
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   157
            self.rollback()
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   158
            raise
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   159
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   160
    # Adjust docstring
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   161
    cmd_process_script.__doc__ = MigrationHelper.cmd_process_script.__doc__
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   162
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   163
    # server specific migration methods ########################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   164
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   165
    def backup_database(self, backupfile=None, askconfirm=True, format='native'):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   166
        config = self.config
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2489
diff changeset
   167
        repo = self.repo_connect()
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   168
        # paths
3192
93c8fdcd943e windows doesn't like ':' in filesc
Aurelien Campeas
parents: 3120
diff changeset
   169
        timestamp = datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   170
        instbkdir = osp.join(config.appdatahome, 'backup')
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   171
        if not osp.exists(instbkdir):
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   172
            os.makedirs(instbkdir)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   173
        backupfile = backupfile or osp.join(instbkdir, '%s-%s.tar.gz'
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   174
                                            % (config.appid, timestamp))
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   175
        # check backup has to be done
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   176
        if osp.exists(backupfile) and not \
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   177
                self.confirm('Backup file %s exists, overwrite it?' % backupfile):
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   178
            print '-> no backup done.'
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   179
            return
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   180
        elif askconfirm and not self.confirm('Backup %s database?' % config.appid):
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   181
            print '-> no backup done.'
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   182
            return
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   183
        open(backupfile,'w').close() # kinda lock
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   184
        os.chmod(backupfile, 0600)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   185
        # backup
6918
9e607157d4cf [cw-ctl] use default `mkdtemp()` behaviour to store backup file internally
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6816
diff changeset
   186
        tmpdir = tempfile.mkdtemp()
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   187
        try:
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   188
            failed = False
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   189
            for source in repo.sources:
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   190
                try:
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   191
                    source.backup(osp.join(tmpdir, source.uri), self.confirm, format=format)
4893
15ae9a33a7f2 [db backup] fix name error in backup_to_file: we've to pass .confirm all along the chain as for restore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   192
                except Exception, ex:
15ae9a33a7f2 [db backup] fix name error in backup_to_file: we've to pass .confirm all along the chain as for restore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   193
                    print '-> error trying to backup %s [%s]' % (source.uri, ex)
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   194
                    if not self.confirm('Continue anyway?', default='n'):
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   195
                        raise SystemExit(1)
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   196
                    else:
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   197
                        failed = True
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   198
            with open(osp.join(tmpdir, 'format.txt'), 'w') as format_file:
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   199
                format_file.write('%s\n' % format)
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   200
            with open(osp.join(tmpdir, 'versions.txt'), 'w') as version_file:
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   201
                versions = repo.get_versions()
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   202
                for cube, version in versions.iteritems():
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   203
                    version_file.write('%s %s\n' % (cube, version))
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   204
            if not failed:
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   205
                bkup = tarfile.open(backupfile, 'w|gz')
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   206
                for filename in os.listdir(tmpdir):
4721
8f63691ccb7f pylint style fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   207
                    bkup.add(osp.join(tmpdir, filename), filename)
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   208
                bkup.close()
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   209
                # call hooks
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   210
                repo.hm.call_hooks('server_backup', repo=repo, timestamp=timestamp)
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   211
                # done
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   212
                print '-> backup file',  backupfile
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   213
        finally:
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   214
            shutil.rmtree(tmpdir)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   215
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2489
diff changeset
   216
    def restore_database(self, backupfile, drop=True, systemonly=True,
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   217
                         askconfirm=True, format='native'):
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   218
        # check
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   219
        if not osp.exists(backupfile):
6323
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   220
            raise ExecutionError("Backup file %s doesn't exist" % backupfile)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   221
        if askconfirm and not self.confirm('Restore %s database from %s ?'
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2926
diff changeset
   222
                                           % (self.config.appid, backupfile)):
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   223
            return
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   224
        # unpack backup
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   225
        tmpdir = tempfile.mkdtemp()
3105
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   226
        try:
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   227
            bkup = tarfile.open(backupfile, 'r|gz')
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   228
        except tarfile.ReadError:
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   229
            # assume restoring old backup
4344
066e7884e57d add source in backup/restore failure message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4339
diff changeset
   230
            shutil.copy(backupfile, osp.join(tmpdir, 'system'))
3105
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   231
        else:
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   232
            for name in bkup.getnames():
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   233
                if name[0] in '/.':
6323
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   234
                    raise ExecutionError('Security check failed, path starts with "/" or "."')
3105
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   235
            bkup.close() # XXX seek error if not close+open !?!
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   236
            bkup = tarfile.open(backupfile, 'r|gz')
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   237
            bkup.extractall(path=tmpdir)
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   238
            bkup.close()
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   239
        if osp.isfile(osp.join(tmpdir, 'format.txt')):
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   240
            with open(osp.join(tmpdir, 'format.txt')) as format_file:
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   241
                written_format = format_file.readline().strip()
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   242
                if written_format in ('portable', 'native'):
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   243
                    format = written_format
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   244
        self.config.init_cnxset_pool = False
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2926
diff changeset
   245
        repo = self.repo_connect()
2861
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   246
        for source in repo.sources:
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   247
            if systemonly and source.uri != 'system':
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   248
                continue
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   249
            try:
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   250
                source.restore(osp.join(tmpdir, source.uri), self.confirm, drop, format)
2861
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   251
            except Exception, exc:
4344
066e7884e57d add source in backup/restore failure message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4339
diff changeset
   252
                print '-> error trying to restore %s [%s]' % (source.uri, exc)
2861
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   253
                if not self.confirm('Continue anyway?', default='n'):
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   254
                    raise SystemExit(1)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   255
        shutil.rmtree(tmpdir)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   256
        # call hooks
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   257
        repo.init_cnxset_pool()
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   258
        repo.hm.call_hooks('server_restore', repo=repo, timestamp=backupfile)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   259
        print '-> database restored.'
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   260
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   261
    @property
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   262
    def cnx(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   263
        """lazy connection"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   264
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   265
            return self._cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   266
        except AttributeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   267
            sourcescfg = self.repo.config.sources()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   268
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   269
                login = sourcescfg['admin']['login']
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   270
                pwd = sourcescfg['admin']['password']
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   271
            except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   272
                login, pwd = manager_userpasswd()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   273
            while True:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   274
                try:
3659
993997b4b41d 3.6 update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3589
diff changeset
   275
                    self._cnx = repo_connect(self.repo, login, password=pwd)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   276
                    if not 'managers' in self._cnx.user(self.session).groups:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   277
                        print 'migration need an account in the managers group'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   278
                    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   279
                        break
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   280
                except AuthenticationError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   281
                    print 'wrong user/password'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   282
                except (KeyboardInterrupt, EOFError):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   283
                    print 'aborting...'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   284
                    sys.exit(0)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   285
                try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   286
                    login, pwd = manager_userpasswd()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   287
                except (KeyboardInterrupt, EOFError):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   288
                    print 'aborting...'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   289
                    sys.exit(0)
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   290
            self.session.keep_cnxset_mode('transaction')
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   291
            self.session.data['rebuild-infered'] = False
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   292
            return self._cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   293
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   294
    @property
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   295
    def session(self):
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   296
        if self.config is not None:
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   297
            session = self.repo._get_session(self.cnx.sessionid)
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   298
            if session.cnxset is None:
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   299
                session.set_read_security(False)
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   300
                session.set_write_security(False)
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   301
            session.set_cnxset()
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   302
            return session
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   303
        # no access to session on remote instance
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   304
        return None
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   305
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   306
    def commit(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   307
        if hasattr(self, '_cnx'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   308
            self._cnx.commit()
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   309
        if self.session:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   310
            self.session.set_cnxset()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   311
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   312
    def rollback(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   313
        if hasattr(self, '_cnx'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
            self._cnx.rollback()
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   315
        if self.session:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   316
            self.session.set_cnxset()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   317
6186
8ada7df95877 [migration] set ask_confirm=False by default on rqlexec
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6179
diff changeset
   318
    def rqlexecall(self, rqliter, ask_confirm=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   319
        for rql, kwargs in rqliter:
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   320
            self.rqlexec(rql, kwargs, ask_confirm=ask_confirm)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   322
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
    def _create_context(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
        """return a dictionary to use as migration script execution context"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
        context = super(ServerMigrationHelper, self)._create_context()
4017
a5b4d4f2a1c7 use commit in migration script instead of checkpoint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4011
diff changeset
   326
        context.update({'commit': self.checkpoint,
4330
cb7b68679501 make rollback available in shell (as commit is)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   327
                        'rollback': self.rollback,
4017
a5b4d4f2a1c7 use commit in migration script instead of checkpoint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4011
diff changeset
   328
                        'checkpoint': deprecated('[3.6] use commit')(self.checkpoint),
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
                        'sql': self.sqlexec,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   330
                        'rql': self.rqlexec,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
                        'rqliter': self.rqliter,
3707
78596919ede3 [c-c] fixes for shell w/ pyro instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3700
diff changeset
   332
                        'schema': self.repo.get_schema(),
78596919ede3 [c-c] fixes for shell w/ pyro instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3700
diff changeset
   333
                        'cnx': self.cnx,
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   334
                        'fsschema': self.fs_schema,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   335
                        'session' : self.session,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   336
                        'repo' : self.repo,
2788
8d3dbe577d3a R put version info in deprecation warnings
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   337
                        'synchronize_schema': deprecated()(self.cmd_sync_schema_props_perms), # 3.4
8d3dbe577d3a R put version info in deprecation warnings
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   338
                        'synchronize_eschema': deprecated()(self.cmd_sync_schema_props_perms), # 3.4
8d3dbe577d3a R put version info in deprecation warnings
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   339
                        'synchronize_rschema': deprecated()(self.cmd_sync_schema_props_perms), # 3.4
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   340
                        })
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   341
        return context
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   342
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   343
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   344
    def group_mapping(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
        """cached group mapping"""
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   346
        return ss.group_mapping(self._cw)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   347
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   348
    @cached
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   349
    def cstrtype_mapping(self):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   350
        """cached constraint types mapping"""
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   351
        return ss.cstrtype_mapping(self._cw)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   352
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   353
    def cmd_exec_event_script(self, event, cube=None, funcname=None,
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   354
                              *args, **kwargs):
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   355
        """execute a cube event scripts  `migration/<event>.py` where event
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   356
        is one of 'precreate', 'postcreate', 'preremove' and 'postremove'.
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   357
        """
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   358
        assert event in ('precreate', 'postcreate', 'preremove', 'postremove')
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   359
        if cube:
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   360
            cubepath = self.config.cube_dir(cube)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   361
            apc = osp.join(cubepath, 'migration', '%s.py' % event)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   362
        else:
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   363
            apc = osp.join(self.config.migration_scripts_dir(), '%s.py' % event)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   364
        if osp.exists(apc):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   365
            if self.config.free_wheel:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2788
diff changeset
   366
                self.cmd_deactivate_verification_hooks()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   367
            self.info('executing %s', apc)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   368
            confirm = self.confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   369
            execscript_confirm = self.execscript_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   370
            self.confirm = yes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   371
            self.execscript_confirm = yes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   372
            try:
3715
e3ccadb126d7 [shell] make process_script available throuhg c-c shell / migration script context
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3708
diff changeset
   373
                return self.cmd_process_script(apc, funcname, *args, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   374
            finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   375
                self.confirm = confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   376
                self.execscript_confirm = execscript_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   377
                if self.config.free_wheel:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2788
diff changeset
   378
                    self.cmd_reactivate_verification_hooks()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   379
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   380
    def cmd_install_custom_sql_scripts(self, cube=None):
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   381
        """install a cube custom sql scripts `schema/*.<driver>.sql` where
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   382
        <driver> depends on the instance main database backend (eg 'postgres',
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   383
        'mysql'...)
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   384
        """
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   385
        driver = self.repo.system_source.dbdriver
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   386
        if cube is None:
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   387
            directory = osp.join(CW_SOFTWARE_ROOT, 'schemas')
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   388
        else:
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   389
            directory = self.config.cube_dir(cube)
7479
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   390
        sql_scripts = []
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   391
        for fpath in glob(osp.join(directory, '*.sql.%s' % driver)):
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   392
            newname = osp.basename(fpath).replace('.sql.%s' % driver,
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   393
                                                  '.%s.sql' % driver)
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   394
            warn('[3.5.6] rename %s into %s' % (fpath, newname),
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   395
                 DeprecationWarning)
7479
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   396
            sql_scripts.append(fpath)
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   397
        sql_scripts += glob(osp.join(directory, '*.%s.sql' % driver))
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   398
        for fpath in sql_scripts:
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   399
            print '-> installing', fpath
7479
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   400
            try:
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   401
                sqlexec(open(fpath).read(), self.session.system_sql, False,
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   402
                        delimiter=';;')
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   403
            except Exception, exc:
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   404
                print '-> ERROR:', exc, ', skipping', fpath
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   405
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   406
    # schema synchronization internals ########################################
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   407
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   408
    def _synchronize_permissions(self, erschema, teid):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   409
        """permission synchronization for an entity or relation type"""
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   410
        assert teid, erschema
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   411
        if 'update' in erschema.ACTIONS or erschema.final:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   412
            # entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   413
            exprtype = u'ERQLExpression'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   414
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   415
            # relation type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   416
            exprtype = u'RRQLExpression'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
        gm = self.group_mapping()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   418
        confirm = self.verbosity >= 2
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   419
        # * remove possibly deprecated permission (eg in the persistent schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   420
        #   but not in the new schema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
        # * synchronize existing expressions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   422
        # * add new groups/expressions
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   423
        for action in erschema.ACTIONS:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   424
            perm = '%s_permission' % action
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   425
            # handle groups
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   426
            newgroups = list(erschema.get_groups(action))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   427
            for geid, gname in self.rqlexec('Any G, GN WHERE T %s G, G name GN, '
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   428
                                            'T eid %%(x)s' % perm, {'x': teid},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   429
                                            ask_confirm=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   430
                if not gname in newgroups:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   431
                    if not confirm or self.confirm('Remove %s permission of %s to %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   432
                                                   % (action, erschema, gname)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   433
                        self.rqlexec('DELETE T %s G WHERE G eid %%(x)s, T eid %s'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   434
                                     % (perm, teid),
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   435
                                     {'x': geid}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   436
                else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   437
                    newgroups.remove(gname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   438
            for gname in newgroups:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   439
                if not confirm or self.confirm('Grant %s permission of %s to %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   440
                                               % (action, erschema, gname)):
6075
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   441
                    try:
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   442
                        self.rqlexec('SET T %s G WHERE G eid %%(x)s, T eid %s'
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   443
                                     % (perm, teid),
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   444
                                     {'x': gm[gname]}, ask_confirm=False)
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   445
                    except KeyError:
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   446
                        self.error('can grant %s perm to unexistant group %s',
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   447
                                   action, gname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   448
            # handle rql expressions
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   449
            newexprs = dict((expr.expression, expr) for expr in erschema.get_rqlexprs(action))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   450
            for expreid, expression in self.rqlexec('Any E, EX WHERE T %s E, E expression EX, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   451
                                                    'T eid %s' % (perm, teid),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   452
                                                    ask_confirm=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   453
                if not expression in newexprs:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   454
                    if not confirm or self.confirm('Remove %s expression for %s permission of %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   455
                                                   % (expression, action, erschema)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   456
                        # deleting the relation will delete the expression entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   457
                        self.rqlexec('DELETE T %s E WHERE E eid %%(x)s, T eid %s'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   458
                                     % (perm, teid),
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   459
                                     {'x': expreid}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   460
                else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   461
                    newexprs.pop(expression)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   462
            for expression in newexprs.values():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   463
                expr = expression.expression
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   464
                if not confirm or self.confirm('Add %s expression for %s permission of %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   465
                                               % (expr, action, erschema)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   466
                    self.rqlexec('INSERT RQLExpression X: X exprtype %%(exprtype)s, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   467
                                 'X expression %%(expr)s, X mainvars %%(vars)s, T %s X '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   468
                                 'WHERE T eid %%(x)s' % perm,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   469
                                 {'expr': expr, 'exprtype': exprtype,
7161
e3f69df8dac7 fix failing tests before 3.12 release
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7116
diff changeset
   470
                                  'vars': u','.join(sorted(expression.mainvars)),
e3f69df8dac7 fix failing tests before 3.12 release
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7116
diff changeset
   471
                                  'x': teid},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   472
                                 ask_confirm=False)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   473
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   474
    def _synchronize_rschema(self, rtype, syncrdefs=True,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   475
                             syncperms=True, syncprops=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   476
        """synchronize properties of the persistent relation schema against its
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   477
        current definition:
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   478
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   479
        * description
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4367
diff changeset
   480
        * symmetric, meta
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   481
        * inlined
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   482
        * relation definitions if `syncrdefs`
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   483
        * permissions if `syncperms`
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   484
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   485
        physical schema changes should be handled by repository's schema hooks
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   486
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   487
        rtype = str(rtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   488
        if rtype in self._synchronized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   489
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   490
        self._synchronized.add(rtype)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   491
        rschema = self.fs_schema.rschema(rtype)
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   492
        reporschema = self.repo.schema.rschema(rtype)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   493
        if syncprops:
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   494
            assert reporschema.eid, reporschema
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   495
            self.rqlexecall(ss.updaterschema2rql(rschema, reporschema.eid),
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   496
                            ask_confirm=self.verbosity>=2)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   497
        if syncrdefs:
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   498
            for subj, obj in rschema.rdefs:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   499
                if (subj, obj) not in reporschema.rdefs:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   500
                    continue
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   501
                if rschema in VIRTUAL_RTYPES:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   502
                    continue
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   503
                self._synchronize_rdef_schema(subj, rschema, obj,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   504
                                              syncprops=syncprops,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   505
                                              syncperms=syncperms)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   506
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   507
    def _synchronize_eschema(self, etype, syncrdefs=True,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   508
                             syncperms=True, syncprops=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   509
        """synchronize properties of the persistent entity schema against
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   510
        its current definition:
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   511
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   512
        * description
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   513
        * internationalizable, fulltextindexed, indexed, meta
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   514
        * relations from/to this entity
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   515
        * __unique_together__
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   516
        * permissions if `syncperms`
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   517
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   518
        etype = str(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   519
        if etype in self._synchronized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   520
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   521
        self._synchronized.add(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   522
        repoeschema = self.repo.schema.eschema(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   523
        try:
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   524
            eschema = self.fs_schema.eschema(etype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   525
        except KeyError:
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   526
            return # XXX somewhat unexpected, no?...
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   527
        if syncprops:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   528
            repospschema = repoeschema.specializes()
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   529
            espschema = eschema.specializes()
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   530
            if repospschema and not espschema:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   531
                self.rqlexec('DELETE X specializes Y WHERE X is CWEType, X name %(x)s',
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   532
                             {'x': str(repoeschema)}, ask_confirm=False)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   533
            elif not repospschema and espschema:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   534
                self.rqlexec('SET X specializes Y WHERE X is CWEType, X name %(x)s, '
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   535
                             'Y is CWEType, Y name %(y)s',
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   536
                             {'x': str(repoeschema), 'y': str(espschema)},
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   537
                             ask_confirm=False)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   538
            self.rqlexecall(ss.updateeschema2rql(eschema, repoeschema.eid),
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   539
                            ask_confirm=self.verbosity >= 2)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   540
        if syncperms:
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   541
            self._synchronize_permissions(eschema, repoeschema.eid)
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   542
        if syncrdefs:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   543
            for rschema, targettypes, role in eschema.relation_definitions(True):
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   544
                if rschema in VIRTUAL_RTYPES:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   545
                    continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   546
                if role == 'subject':
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   547
                    if not rschema in repoeschema.subject_relations():
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   548
                        continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   549
                    subjtypes, objtypes = [etype], targettypes
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   550
                else: # role == 'object'
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   551
                    if not rschema in repoeschema.object_relations():
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   552
                        continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   553
                    subjtypes, objtypes = targettypes, [etype]
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   554
                self._synchronize_rschema(rschema, syncrdefs=False,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   555
                                          syncprops=syncprops, syncperms=syncperms)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   556
                reporschema = self.repo.schema.rschema(rschema)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   557
                for subj in subjtypes:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   558
                    for obj in objtypes:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   559
                        if (subj, obj) not in reporschema.rdefs:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   560
                            continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   561
                        self._synchronize_rdef_schema(subj, rschema, obj,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   562
                                                      syncprops=syncprops, syncperms=syncperms)
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   563
        if syncprops: # need to process __unique_together__ after rdefs were processed
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   564
            repo_unique_together = set([frozenset(ut)
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   565
                                        for ut in repoeschema._unique_together])
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   566
            unique_together = set([frozenset(ut)
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   567
                                   for ut in eschema._unique_together])
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   568
            for ut in repo_unique_together - unique_together:
6815
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   569
                restrictions = []
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   570
                substs = {'x': repoeschema.eid}
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   571
                for i, col in enumerate(ut):
6815
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   572
                    restrictions.append('C relations T%(i)d, '
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   573
                                       'T%(i)d name %%(T%(i)d)s' % {'i': i})
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   574
                    substs['T%d'%i] = col
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   575
                self.rqlexec('DELETE CWUniqueTogetherConstraint C '
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   576
                             'WHERE C constraint_of E, '
6815
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   577
                             '      E eid %%(x)s,'
7008
5d0be45ee69d cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6918
diff changeset
   578
                             '      %s' % ', '.join(restrictions),
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   579
                             substs)
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   580
            for ut in unique_together - repo_unique_together:
6815
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   581
                rql, substs = ss.uniquetogether2rql(eschema, ut)
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   582
                substs['x'] = repoeschema.eid
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   583
                self.rqlexec(rql, substs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   584
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   585
    def _synchronize_rdef_schema(self, subjtype, rtype, objtype,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   586
                                 syncperms=True, syncprops=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   587
        """synchronize properties of the persistent relation definition schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   588
        against its current definition:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   589
        * order and other properties
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   590
        * constraints
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   591
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   592
        subjtype, objtype = str(subjtype), str(objtype)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   593
        rschema = self.fs_schema.rschema(rtype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   594
        reporschema = self.repo.schema.rschema(rschema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   595
        if (subjtype, rschema, objtype) in self._synchronized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   596
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   597
        self._synchronized.add((subjtype, rschema, objtype))
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4367
diff changeset
   598
        if rschema.symmetric:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   599
            self._synchronized.add((objtype, rschema, subjtype))
4949
f4dce73da26b [schema sync] don't try to synchronize infered relation defs, fixing a name error on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4947
diff changeset
   600
        rdef = rschema.rdef(subjtype, objtype)
f4dce73da26b [schema sync] don't try to synchronize infered relation defs, fixing a name error on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4947
diff changeset
   601
        if rdef.infered:
f4dce73da26b [schema sync] don't try to synchronize infered relation defs, fixing a name error on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4947
diff changeset
   602
            return # don't try to synchronize infered relation defs
4950
bca0873c0d6e [schema sync] fix another potential name error on synchronizing rdefs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4949
diff changeset
   603
        repordef = reporschema.rdef(subjtype, objtype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   604
        confirm = self.verbosity >= 2
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   605
        if syncprops:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   606
            # properties
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   607
            self.rqlexecall(ss.updaterdef2rql(rdef, repordef.eid),
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   608
                            ask_confirm=confirm)
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   609
            # constraints
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   610
            newconstraints = list(rdef.constraints)
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   611
            # 1. remove old constraints and update constraints of the same type
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   612
            # NOTE: don't use rschema.constraint_by_type because it may be
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   613
            #       out of sync with newconstraints when multiple
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   614
            #       constraints of the same type are used
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   615
            for cstr in repordef.constraints:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   616
                for newcstr in newconstraints:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   617
                    if newcstr.type() == cstr.type():
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   618
                        break
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   619
                else:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   620
                    newcstr = None
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   621
                if newcstr is None:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   622
                    self.rqlexec('DELETE X constrained_by C WHERE C eid %(x)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   623
                                 {'x': cstr.eid}, ask_confirm=confirm)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   624
                else:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   625
                    newconstraints.remove(newcstr)
4947
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   626
                    value = unicode(newcstr.serialize())
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   627
                    if value != unicode(cstr.serialize()):
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   628
                        self.rqlexec('SET X value %(v)s WHERE X eid %(x)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   629
                                     {'x': cstr.eid, 'v': value},
4947
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   630
                                     ask_confirm=confirm)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   631
            # 2. add new constraints
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   632
            cstrtype_map = self.cstrtype_mapping()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   633
            self.rqlexecall(ss.constraints2rql(cstrtype_map, newconstraints,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   634
                                               repordef.eid),
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   635
                            ask_confirm=confirm)
4188
b3258d2afe04 fix virtual relation detection: erschema is no more a relation type but a relation def in _synchronize_permissions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4099
diff changeset
   636
        if syncperms and not rschema in VIRTUAL_RTYPES:
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   637
            self._synchronize_permissions(rdef, repordef.eid)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   638
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   639
    # base actions ############################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   640
4011
394f853bb653 [migration] write migration instructions for permissions handling on relation definition
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3998
diff changeset
   641
    def checkpoint(self, ask_confirm=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   642
        """checkpoint action"""
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   643
        if not ask_confirm or self.confirm('Commit now ?', shell=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   644
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   645
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   646
    def cmd_add_cube(self, cube, update_database=True):
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   647
        self.cmd_add_cubes( (cube,), update_database)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   648
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   649
    def cmd_add_cubes(self, cubes, update_database=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   650
        """update_database is telling if the database schema should be updated
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   651
        or if only the relevant eproperty should be inserted (for the case where
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2275
diff changeset
   652
        a cube has been extracted from an existing instance, so the
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   653
        cube schema is already in there)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   654
        """
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   655
        newcubes = super(ServerMigrationHelper, self).cmd_add_cubes(cubes)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   656
        if not newcubes:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   657
            return
2107
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   658
        for cube in newcubes:
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   659
            self.cmd_set_property('system.version.'+cube,
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   660
                                  self.config.cube_version(cube))
3836
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   661
            # ensure added cube is in config cubes
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   662
            # XXX worth restoring on error?
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   663
            if not cube in self.config._cubes:
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   664
                self.config._cubes += (cube,)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   665
        if not update_database:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   666
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   667
            return
1036
593df4919845 when reading the schema while adding/removing cubes, read schema in non-strict mode
sylvain.thenault@logilab.fr
parents: 1016
diff changeset
   668
        newcubes_schema = self.config.load_schema(construction_mode='non-strict')
3836
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   669
        # XXX we have to replace fs_schema, used in cmd_add_relation_type
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   670
        # etc. and fsschema of migration script contexts
6217
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6208
diff changeset
   671
        self.fs_schema = newcubes_schema
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6208
diff changeset
   672
        self.update_context('fsschema', self.fs_schema)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   673
        new = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   674
        # execute pre-create files
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   675
        driver = self.repo.system_source.dbdriver
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   676
        for cube in reversed(newcubes):
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   677
            self.cmd_install_custom_sql_scripts(cube)
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   678
            self.cmd_exec_event_script('precreate', cube)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   679
        # add new entity and relation types
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   680
        for rschema in newcubes_schema.relations():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   681
            if not rschema in self.repo.schema:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   682
                self.cmd_add_relation_type(rschema.type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   683
                new.add(rschema.type)
2926
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   684
        toadd = [eschema for eschema in newcubes_schema.entities()
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   685
                 if not eschema in self.repo.schema]
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   686
        for eschema in order_eschemas(toadd):
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   687
            self.cmd_add_entity_type(eschema.type)
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   688
            new.add(eschema.type)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   689
        # check if attributes has been added to existing entities
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   690
        for rschema in newcubes_schema.relations():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   691
            existingschema = self.repo.schema.rschema(rschema.type)
4099
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   692
            for (fromtype, totype) in rschema.rdefs:
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   693
                if (fromtype, totype) in existingschema.rdefs:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   694
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   695
                # check we should actually add the relation definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   696
                if not (fromtype in new or totype in new or rschema in new):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   697
                    continue
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   698
                self.cmd_add_relation_definition(str(fromtype), rschema.type,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   699
                                                 str(totype))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   700
        # execute post-create files
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   701
        for cube in reversed(newcubes):
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   702
            self.cmd_exec_event_script('postcreate', cube)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   703
            self.commit()
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   704
2124
5a0b02f37b23 set removedeps to False by default, raise an exception instead of a simple assertion for error, more remove_cube tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2122
diff changeset
   705
    def cmd_remove_cube(self, cube, removedeps=False):
2122
4ea13a828513 add removedeps option to remove_cube to control wether a cube's dependencies should be removed as well or not
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2107
diff changeset
   706
        removedcubes = super(ServerMigrationHelper, self).cmd_remove_cube(
4ea13a828513 add removedeps option to remove_cube to control wether a cube's dependencies should be removed as well or not
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2107
diff changeset
   707
            cube, removedeps)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   708
        if not removedcubes:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   709
            return
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   710
        fsschema = self.fs_schema
1036
593df4919845 when reading the schema while adding/removing cubes, read schema in non-strict mode
sylvain.thenault@logilab.fr
parents: 1016
diff changeset
   711
        removedcubes_schema = self.config.load_schema(construction_mode='non-strict')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   712
        reposchema = self.repo.schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   713
        # execute pre-remove files
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   714
        for cube in reversed(removedcubes):
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   715
            self.cmd_exec_event_script('preremove', cube)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   716
        # remove cubes'entity and relation types
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   717
        for rschema in fsschema.relations():
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   718
            if not rschema in removedcubes_schema and rschema in reposchema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   719
                self.cmd_drop_relation_type(rschema.type)
2926
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   720
        toremove = [eschema for eschema in fsschema.entities()
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   721
                    if not eschema in removedcubes_schema
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   722
                    and eschema in reposchema]
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   723
        for eschema in reversed(order_eschemas(toremove)):
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   724
            self.cmd_drop_entity_type(eschema.type)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   725
        for rschema in fsschema.relations():
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   726
            if rschema in removedcubes_schema and rschema in reposchema:
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   727
                # check if attributes/relations has been added to entities from
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   728
                # other cubes
4099
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   729
                for fromtype, totype in rschema.rdefs:
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   730
                    if (fromtype, totype) not in removedcubes_schema[rschema.type].rdefs and \
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   731
                           (fromtype, totype) in reposchema[rschema.type].rdefs:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   732
                        self.cmd_drop_relation_definition(
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   733
                            str(fromtype), rschema.type, str(totype))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   734
        # execute post-remove files
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   735
        for cube in reversed(removedcubes):
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   736
            self.cmd_exec_event_script('postremove', cube)
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   737
            self.rqlexec('DELETE CWProperty X WHERE X pkey %(pk)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   738
                         {'pk': u'system.version.'+cube}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   739
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   740
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   741
    # schema migration actions ################################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   742
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   743
    def cmd_add_attribute(self, etype, attrname, attrtype=None, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   744
        """add a new attribute on the given entity type"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   745
        if attrtype is None:
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   746
            rschema = self.fs_schema.rschema(attrname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   747
            attrtype = rschema.objects(etype)[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   748
        self.cmd_add_relation_definition(etype, attrname, attrtype, commit=commit)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   749
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   750
    def cmd_drop_attribute(self, etype, attrname, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   751
        """drop an existing attribute from the given entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   752
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   753
        `attrname` is a string giving the name of the attribute to drop
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   754
        """
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   755
        try:
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   756
            rschema = self.repo.schema.rschema(attrname)
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   757
            attrtype = rschema.objects(etype)[0]
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   758
        except KeyError:
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   759
            print 'warning: attribute %s %s is not known, skip deletion' % (
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   760
                etype, attrname)
7558
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   761
        else:
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   762
            self.cmd_drop_relation_definition(etype, attrname, attrtype,
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   763
                                              commit=commit)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   764
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   765
    def cmd_rename_attribute(self, etype, oldname, newname, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   766
        """rename an existing attribute of the given entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   767
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   768
        `oldname` is a string giving the name of the existing attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   769
        `newname` is a string giving the name of the renamed attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   770
        """
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   771
        eschema = self.fs_schema.eschema(etype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   772
        attrtype = eschema.destination(newname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   773
        # have to commit this first step anyway to get the definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   774
        # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   775
        self.cmd_add_attribute(etype, newname, attrtype, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   776
        # skipp NULL values if the attribute is required
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   777
        rql = 'SET X %s VAL WHERE X is %s, X %s VAL' % (newname, etype, oldname)
4633
a85b4361fb22 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4580
diff changeset
   778
        card = eschema.rdef(newname).cardinality[0]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   779
        if card == '1':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   780
            rql += ', NOT X %s NULL' % oldname
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   781
        self.rqlexec(rql, ask_confirm=self.verbosity>=2)
3548
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   782
        # XXX if both attributes fulltext indexed, should skip fti rebuild
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   783
        # XXX if old attribute was fti indexed but not the new one old value
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   784
        # won't be removed from the index (this occurs on other kind of
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   785
        # fulltextindexed change...)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   786
        self.cmd_drop_attribute(etype, oldname, commit=commit)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   787
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   788
    def cmd_add_entity_type(self, etype, auto=True, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   789
        """register a new entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   790
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   791
        in auto mode, automatically register entity's relation where the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   792
        targeted type is known
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   793
        """
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   794
        instschema = self.repo.schema
7166
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7161
diff changeset
   795
        eschema = self.fs_schema.eschema(etype)
7558
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   796
        if etype in instschema and not (eschema.final and eschema.eid is None):
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   797
            print 'warning: %s already known, skip addition' % etype
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   798
            return
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   799
        confirm = self.verbosity >= 2
4043
39ae94e0c8b8 give group mapping where needed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4042
diff changeset
   800
        groupmap = self.group_mapping()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   801
        cstrtypemap = self.cstrtype_mapping()
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   802
        # register the entity into CWEType
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   803
        execute = self._cw.execute
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   804
        ss.execschemarql(execute, eschema, ss.eschema2rql(eschema, groupmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   805
        # add specializes relation if needed
5293
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   806
        specialized = eschema.specializes()
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   807
        if specialized:
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   808
            try:
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   809
                specialized.eid = instschema[specialized].eid
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   810
            except KeyError:
6323
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   811
                raise ExecutionError('trying to add entity type but parent type is '
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   812
                                     'not yet in the database schema')
5293
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   813
            self.rqlexecall(ss.eschemaspecialize2rql(eschema), ask_confirm=confirm)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   814
        # register entity's attributes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   815
        for rschema, attrschema in eschema.attribute_definitions():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   816
            # ignore those meta relations, they will be automatically added
2617
89c62b855f2e [R schema hooks] rename META_RELATIONS_TYPES into META_RTYPES, use it and other schema consts to avoid errors
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2616
diff changeset
   817
            if rschema.type in META_RTYPES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   818
                continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   819
            if not rschema.type in instschema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   820
                # need to add the relation type and to commit to get it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   821
                # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   822
                self.cmd_add_relation_type(rschema.type, False, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   823
            # register relation definition
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   824
            rdef = self._get_rdef(rschema, eschema, eschema.destination(rschema))
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   825
            ss.execschemarql(execute, rdef, ss.rdef2rql(rdef, cstrtypemap, groupmap),)
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   826
        # take care to newly introduced base class
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   827
        # XXX some part of this should probably be under the "if auto" block
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   828
        for spschema in eschema.specialized_by(recursive=False):
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   829
            try:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   830
                instspschema = instschema[spschema]
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   831
            except KeyError:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   832
                # specialized entity type not in schema, ignore
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   833
                continue
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   834
            if instspschema.specializes() != eschema:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   835
                self.rqlexec('SET D specializes P WHERE D eid %(d)s, P name %(pn)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   836
                             {'d': instspschema.eid, 'pn': eschema.type},
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   837
                             ask_confirm=confirm)
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   838
                for rschema, tschemas, role in spschema.relation_definitions(True):
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   839
                    for tschema in tschemas:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   840
                        if not tschema in instschema:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   841
                            continue
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   842
                        if role == 'subject':
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   843
                            subjschema = spschema
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   844
                            objschema = tschema
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3582
diff changeset
   845
                            if rschema.final and rschema in instspschema.subjrels:
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   846
                                # attribute already set, has_rdef would check if
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   847
                                # it's of the same type, we don't want this so
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   848
                                # simply skip here
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   849
                                continue
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   850
                        elif role == 'object':
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   851
                            subjschema = tschema
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   852
                            objschema = spschema
4633
a85b4361fb22 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4580
diff changeset
   853
                        if (rschema.rdef(subjschema, objschema).infered
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   854
                            or (instschema.has_relation(rschema) and
4633
a85b4361fb22 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4580
diff changeset
   855
                                (subjschema, objschema) in instschema[rschema].rdefs)):
4721
8f63691ccb7f pylint style fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   856
                            continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   857
                        self.cmd_add_relation_definition(
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   858
                            subjschema.type, rschema.type, objschema.type)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   859
        if auto:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   860
            # we have commit here to get relation types actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   861
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   862
            added = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   863
            for rschema in eschema.subject_relations():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   864
                # attribute relation have already been processed and
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   865
                # 'owned_by'/'created_by' will be automatically added
2617
89c62b855f2e [R schema hooks] rename META_RELATIONS_TYPES into META_RTYPES, use it and other schema consts to avoid errors
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2616
diff changeset
   866
                if rschema.final or rschema.type in META_RTYPES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   867
                    continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   868
                rtypeadded = rschema.type in instschema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   869
                for targetschema in rschema.objects(etype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   870
                    # ignore relations where the targeted type is not in the
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2275
diff changeset
   871
                    # current instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   872
                    targettype = targetschema.type
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   873
                    if not targettype in instschema and targettype != etype:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   874
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   875
                    if not rtypeadded:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   876
                        # need to add the relation type and to commit to get it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   877
                        # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   878
                        added.append(rschema.type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   879
                        self.cmd_add_relation_type(rschema.type, False, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   880
                        rtypeadded = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   881
                    # register relation definition
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4367
diff changeset
   882
                    # remember this two avoid adding twice non symmetric relation
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   883
                    # such as "Emailthread forked_from Emailthread"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   884
                    added.append((etype, rschema.type, targettype))
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   885
                    rdef = self._get_rdef(rschema, eschema, targetschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   886
                    ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   887
                                     ss.rdef2rql(rdef, cstrtypemap, groupmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   888
            for rschema in eschema.object_relations():
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   889
                if rschema.type in META_RTYPES:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   890
                    continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   891
                rtypeadded = rschema.type in instschema or rschema.type in added
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   892
                for targetschema in rschema.subjects(etype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   893
                    # ignore relations where the targeted type is not in the
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2275
diff changeset
   894
                    # current instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   895
                    targettype = targetschema.type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   896
                    # don't check targettype != etype since in this case the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   897
                    # relation has already been added as a subject relation
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   898
                    if not targettype in instschema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   899
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   900
                    if not rtypeadded:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   901
                        # need to add the relation type and to commit to get it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   902
                        # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   903
                        self.cmd_add_relation_type(rschema.type, False, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   904
                        rtypeadded = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   905
                    elif (targettype, rschema.type, etype) in added:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   906
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   907
                    # register relation definition
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   908
                    rdef = self._get_rdef(rschema, targetschema, eschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   909
                    ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   910
                                     ss.rdef2rql(rdef, cstrtypemap, groupmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   911
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   912
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   913
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   914
    def cmd_drop_entity_type(self, etype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   915
        """unregister an existing entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   916
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   917
        This will trigger deletion of necessary relation types and definitions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   918
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   919
        # XXX what if we delete an entity type which is specialized by other types
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   920
        # unregister the entity from CWEType
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   921
        self.rqlexec('DELETE CWEType X WHERE X name %(etype)s', {'etype': etype},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   922
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   923
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   924
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   925
6115
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   926
    def cmd_rename_entity_type(self, oldname, newname, attrs=None, commit=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   927
        """rename an existing entity type in the persistent schema
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   928
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   929
        `oldname` is a string giving the name of the existing entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   930
        `newname` is a string giving the name of the renamed entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   931
        """
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   932
        schema = self.repo.schema
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   933
        if newname in schema:
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   934
            assert oldname in ETYPE_NAME_MAP, \
6115
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   935
                   '%s should be mapped to %s in ETYPE_NAME_MAP' % (oldname,
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   936
                                                                    newname)
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   937
            if attrs is None:
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   938
                attrs = ','.join(SQL_PREFIX + rschema.type
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   939
                                 for rschema in schema[newname].subject_relations()
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   940
                                 if (rschema.final or rschema.inlined)
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   941
                                 and not rschema in PURE_VIRTUAL_RTYPES)
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   942
            else:
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   943
                attrs += ('eid', 'creation_date', 'modification_date', 'cwuri')
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   944
                attrs = ','.join(SQL_PREFIX + attr for attr in attrs)
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   945
            self.sqlexec('INSERT INTO %s%s(%s) SELECT %s FROM %s%s' % (
5999
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   946
                SQL_PREFIX, newname, attrs, attrs, SQL_PREFIX, oldname),
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   947
                         ask_confirm=False)
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   948
            # old entity type has not been added to the schema, can't gather it
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   949
            new = schema.eschema(newname)
6117
f3fdfcbda54b cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6116
diff changeset
   950
            oldeid = self.rqlexec('CWEType ET WHERE ET name %(on)s',
f3fdfcbda54b cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6116
diff changeset
   951
                                  {'on': oldname}, ask_confirm=False)[0][0]
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   952
            # backport old type relations to new type
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   953
            # XXX workflows, other relations?
5999
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   954
            for r1, rr1 in [('from_entity', 'to_entity'),
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   955
                            ('to_entity', 'from_entity')]:
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   956
                self.rqlexec('SET X %(r1)s NET WHERE X %(r1)s OET, '
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   957
                             'NOT EXISTS(X2 %(r1)s NET, X relation_type XRT, '
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   958
                             'X2 relation_type XRT, X %(rr1)s XTE, X2 %(rr1)s XTE), '
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   959
                             'OET eid %%(o)s, NET eid %%(n)s' % locals(),
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   960
                             {'o': oldeid, 'n': new.eid}, ask_confirm=False)
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   961
            # backport is / is_instance_of relation to new type
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   962
            for rtype in ('is', 'is_instance_of'):
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   963
                self.sqlexec('UPDATE %s_relation SET eid_to=%s WHERE eid_to=%s'
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   964
                             % (rtype, new.eid, oldeid), ask_confirm=False)
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   965
            # delete relations using SQL to avoid relations content removal
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   966
            # triggered by schema synchronization hooks.
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   967
            session = self.session
6116
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   968
            for rdeftype in ('CWRelation', 'CWAttribute'):
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   969
                thispending = set( (eid for eid, in self.sqlexec(
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   970
                    'SELECT cw_eid FROM cw_%s WHERE cw_from_entity=%%(eid)s OR '
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   971
                    ' cw_to_entity=%%(eid)s' % rdeftype,
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   972
                    {'eid': oldeid}, ask_confirm=False)) )
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   973
                # we should add deleted eids into pending eids else we may
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   974
                # get some validation error on commit since integrity hooks
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   975
                # may think some required relation is missing... This also ensure
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   976
                # repository caches are properly cleanup
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   977
                hook.CleanupDeletedEidsCacheOp.get_instance(session).union(thispending)
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   978
                # and don't forget to remove record from system tables
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   979
                entities = [session.entity_from_eid(eid, rdeftype) for eid in thispending]
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   980
                self.repo.system_source.delete_info_multi(session, entities, 'system')
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   981
                self.sqlexec('DELETE FROM cw_%s WHERE cw_from_entity=%%(eid)s OR '
6116
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   982
                             'cw_to_entity=%%(eid)s' % rdeftype,
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   983
                             {'eid': oldeid}, ask_confirm=False)
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   984
                # now we have to manually cleanup relations pointing to deleted
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   985
                # entities
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   986
                thiseids = ','.join(str(eid) for eid in thispending)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   987
                for rschema, ttypes, role in schema[rdeftype].relation_definitions():
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   988
                    if rschema.type in VIRTUAL_RTYPES:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   989
                        continue
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   990
                    sqls = []
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   991
                    if role == 'object':
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   992
                        if rschema.inlined:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   993
                            for eschema in ttypes:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   994
                                sqls.append('DELETE FROM cw_%s WHERE cw_%s IN(%%s)'
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   995
                                            % (eschema, rschema))
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   996
                        else:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   997
                            sqls.append('DELETE FROM %s_relation WHERE eid_to IN(%%s)'
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   998
                                        % rschema)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   999
                    elif not rschema.inlined:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1000
                        sqls.append('DELETE FROM %s_relation WHERE eid_from IN(%%s)'
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1001
                                    % rschema)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1002
                    for sql in sqls:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
  1003
                        self.sqlexec(sql % thiseids, ask_confirm=False)
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
  1004
            # remove the old type: use rql to propagate deletion
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
  1005
            self.rqlexec('DELETE CWEType ET WHERE ET name %(on)s', {'on': oldname},
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
  1006
                         ask_confirm=False)
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1007
        else:
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
  1008
            self.rqlexec('SET ET name %(newname)s WHERE ET is CWEType, ET name %(on)s',
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
  1009
                         {'newname' : unicode(newname), 'on' : oldname},
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1010
                         ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1011
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1012
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1013
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1014
    def cmd_add_relation_type(self, rtype, addrdef=True, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1015
        """register a new relation type named `rtype`, as described in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1016
        schema description file.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1017
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1018
        `addrdef` is a boolean value; when True, it will also add all relations
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1019
        of the type just added found in the schema definition file. Note that it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1020
        implies an intermediate "commit" which commits the relation type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1021
        creation (but not the relation definitions themselves, for which
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1022
        committing depends on the `commit` argument value).
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1023
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1024
        """
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1025
        reposchema = self.repo.schema
7558
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
  1026
        if rtype in reposchema:
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1027
            print 'warning: relation type %s is already known, skip addition' % (
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1028
                rtype)
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1029
            return
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
  1030
        rschema = self.fs_schema.rschema(rtype)
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1031
        execute = self._cw.execute
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1032
        # register the relation into CWRType and insert necessary relation
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1033
        # definitions
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1034
        ss.execschemarql(execute, rschema, ss.rschema2rql(rschema, addrdef=False))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1035
        if addrdef:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1036
            self.commit()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1037
            gmap = self.group_mapping()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1038
            cmap = self.cstrtype_mapping()
6073
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1039
            done = set()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1040
            for rdef in rschema.rdefs.itervalues():
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1041
                if not (reposchema.has_entity(rdef.subject)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1042
                        and reposchema.has_entity(rdef.object)):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1043
                    continue
6073
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1044
                # symmetric relations appears twice
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1045
                if (rdef.subject, rdef.object) in done:
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1046
                    continue
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1047
                done.add( (rdef.subject, rdef.object) )
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1048
                self._set_rdef_eid(rdef)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1049
                ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1050
                                 ss.rdef2rql(rdef, cmap, gmap))
2700
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1051
            if rtype in META_RTYPES:
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1052
                # if the relation is in META_RTYPES, ensure we're adding it for
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1053
                # all entity types *in the persistent schema*, not only those in
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1054
                # the fs schema
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1055
                for etype in self.repo.schema.entities():
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1056
                    if not etype in self.fs_schema:
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1057
                        # get sample object type and rproperties
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1058
                        objtypes = rschema.objects()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1059
                        assert len(objtypes) == 1, objtypes
2700
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1060
                        objtype = objtypes[0]
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1061
                        rdef = copy(rschema.rdef(rschema.subjects(objtype)[0], objtype))
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1062
                        rdef.subject = etype
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1063
                        rdef.rtype = self.repo.schema.rschema(rschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1064
                        rdef.object = self.repo.schema.rschema(objtype)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1065
                        ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1066
                                         ss.rdef2rql(rdef, cmap, gmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1067
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1068
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1069
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1070
    def cmd_drop_relation_type(self, rtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1071
        """unregister an existing relation type"""
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1072
        # unregister the relation from CWRType
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1073
        self.rqlexec('DELETE CWRType X WHERE X name %r' % rtype,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1074
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1075
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1076
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1077
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1078
    def cmd_rename_relation(self, oldname, newname, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1079
        """rename an existing relation
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1080
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1081
        `oldname` is a string giving the name of the existing relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1082
        `newname` is a string giving the name of the renamed relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1083
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1084
        self.cmd_add_relation_type(newname, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1085
        self.rqlexec('SET X %s Y WHERE X %s Y' % (newname, oldname),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1086
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1087
        self.cmd_drop_relation_type(oldname, commit=commit)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1088
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1089
    def cmd_add_relation_definition(self, subjtype, rtype, objtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1090
        """register a new relation definition, from its definition found in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1091
        schema definition file
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1092
        """
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
  1093
        rschema = self.fs_schema.rschema(rtype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1094
        if not rtype in self.repo.schema:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1095
            self.cmd_add_relation_type(rtype, addrdef=False, commit=True)
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1096
        if (subjtype, objtype) in self.repo.schema.rschema(rtype).rdefs:
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1097
            print 'warning: relation %s %s %s is already known, skip addition' % (
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1098
                subjtype, rtype, objtype)
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1099
            return
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1100
        execute = self._cw.execute
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1101
        rdef = self._get_rdef(rschema, subjtype, objtype)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1102
        ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1103
                         ss.rdef2rql(rdef, self.cstrtype_mapping(),
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1104
                                     self.group_mapping()))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1105
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1106
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1107
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1108
    def _get_rdef(self, rschema, subjtype, objtype):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1109
        return self._set_rdef_eid(rschema.rdefs[(subjtype, objtype)])
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1110
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1111
    def _set_rdef_eid(self, rdef):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1112
        for attr in ('rtype', 'subject', 'object'):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1113
            schemaobj = getattr(rdef, attr)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1114
            if getattr(schemaobj, 'eid', None) is None:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1115
                schemaobj.eid =  self.repo.schema[schemaobj].eid
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1116
                assert schemaobj.eid is not None
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1117
        return rdef
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1118
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1119
    def cmd_drop_relation_definition(self, subjtype, rtype, objtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1120
        """unregister an existing relation definition"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1121
        rschema = self.repo.schema.rschema(rtype)
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1122
        # unregister the definition from CWAttribute or CWRelation
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3582
diff changeset
  1123
        if rschema.final:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1124
            etype = 'CWAttribute'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1125
        else:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1126
            etype = 'CWRelation'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1127
        rql = ('DELETE %s X WHERE X from_entity FE, FE name "%s",'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1128
               'X relation_type RT, RT name "%s", X to_entity TE, TE name "%s"')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1129
        self.rqlexec(rql % (etype, subjtype, rtype, objtype),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1130
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1131
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1132
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1133
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1134
    def cmd_sync_schema_props_perms(self, ertype=None, syncperms=True,
1409
f4dee84a618f sql attributes bugfix
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 1399
diff changeset
  1135
                                    syncprops=True, syncrdefs=True, commit=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1136
        """synchronize the persistent schema against the current definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1137
        schema.
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1138
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1139
        `ertype` can be :
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1140
        - None, in that case everything will be synced ;
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1141
        - a string, it should be an entity type or
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1142
          a relation type. In that case, only the corresponding
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1143
          entities / relations will be synced ;
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1144
        - an rdef object to synchronize only this specific relation definition
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1145
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1146
        It will synch common stuff between the definition schema and the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1147
        actual persistent schema, it won't add/remove any entity or relation.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1148
        """
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1149
        assert syncperms or syncprops, 'nothing to do'
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1150
        if ertype is not None:
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1151
            if isinstance(ertype, RelationDefinitionSchema):
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1152
                ertype = ertype.as_triple()
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1153
            if isinstance(ertype, (tuple, list)):
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1154
                assert len(ertype) == 3, 'not a relation definition'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
  1155
                self._synchronize_rdef_schema(ertype[0], ertype[1], ertype[2],
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1156
                                              syncperms=syncperms,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1157
                                              syncprops=syncprops)
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1158
            else:
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1159
                erschema = self.repo.schema[ertype]
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1160
                if isinstance(erschema, CubicWebRelationSchema):
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1161
                    self._synchronize_rschema(erschema, syncrdefs=syncrdefs,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1162
                                              syncperms=syncperms,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1163
                                              syncprops=syncprops)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1164
                else:
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1165
                    self._synchronize_eschema(erschema, syncrdefs=syncrdefs,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1166
                                              syncperms=syncperms,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1167
                                              syncprops=syncprops)
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1168
        else:
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1169
            for etype in self.repo.schema.entities():
7505
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1170
                if etype.eid is None:
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1171
                     # not yet added final etype (thing to BigInt defined in
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1172
                     # yams though 3.13 migration not done yet)
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1173
                    continue
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1174
                self._synchronize_eschema(etype, syncrdefs=syncrdefs,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1175
                                          syncprops=syncprops, syncperms=syncperms)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1176
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1177
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1178
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1179
    def cmd_change_relation_props(self, subjtype, rtype, objtype,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1180
                                  commit=True, **kwargs):
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1181
        """change some properties of a relation definition
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1182
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1183
        you usually want to use sync_schema_props_perms instead.
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1184
        """
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1185
        assert kwargs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1186
        restriction = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1187
        if subjtype and subjtype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1188
            restriction.append('X from_entity FE, FE name "%s"' % subjtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1189
        if objtype and objtype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1190
            restriction.append('X to_entity TE, TE name "%s"' % objtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1191
        if rtype and rtype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1192
            restriction.append('X relation_type RT, RT name "%s"' % rtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1193
        assert restriction
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1194
        values = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1195
        for k, v in kwargs.items():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1196
            values.append('X %s %%(%s)s' % (k, k))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1197
            if isinstance(v, str):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1198
                kwargs[k] = unicode(v)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1199
        rql = 'SET %s WHERE %s' % (','.join(values), ','.join(restriction))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1200
        self.rqlexec(rql, kwargs, ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1201
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1202
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1203
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1204
    def cmd_set_size_constraint(self, etype, rtype, size, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1205
        """set change size constraint of a string attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1206
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1207
        if size is None any size constraint will be removed.
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1208
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1209
        you usually want to use sync_schema_props_perms instead.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1210
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1211
        oldvalue = None
4695
4aaf87e7f79e 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4650
diff changeset
  1212
        for constr in self.repo.schema.eschema(etype).rdef(rtype).constraints:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1213
            if isinstance(constr, SizeConstraint):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1214
                oldvalue = constr.max
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1215
        if oldvalue == size:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1216
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1217
        if oldvalue is None and not size is None:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1218
            ceid = self.rqlexec('INSERT CWConstraint C: C value %(v)s, C cstrtype CT '
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1219
                                'WHERE CT name "SizeConstraint"',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1220
                                {'v': SizeConstraint(size).serialize()},
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1221
                                ask_confirm=self.verbosity>=2)[0][0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1222
            self.rqlexec('SET X constrained_by C WHERE X from_entity S, X relation_type R, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1223
                         'S name "%s", R name "%s", C eid %s' % (etype, rtype, ceid),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1224
                         ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1225
        elif not oldvalue is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1226
            if not size is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1227
                self.rqlexec('SET C value %%(v)s WHERE X from_entity S, X relation_type R,'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1228
                             'X constrained_by C, C cstrtype CT, CT name "SizeConstraint",'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1229
                             'S name "%s", R name "%s"' % (etype, rtype),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1230
                             {'v': unicode(SizeConstraint(size).serialize())},
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1231
                             ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1232
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1233
                self.rqlexec('DELETE X constrained_by C WHERE X from_entity S, X relation_type R,'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1234
                             'X constrained_by C, C cstrtype CT, CT name "SizeConstraint",'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1235
                             'S name "%s", R name "%s"' % (etype, rtype),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1236
                             ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1237
                # cleanup unused constraints
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1238
                self.rqlexec('DELETE CWConstraint C WHERE NOT X constrained_by C')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1239
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1240
            self.commit()
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1241
3577
067610a0e12f add version number to deprecation messages
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3548
diff changeset
  1242
    @deprecated('[3.2] use sync_schema_props_perms(ertype, syncprops=False)')
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1243
    def cmd_synchronize_permissions(self, ertype, commit=True):
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1244
        self.cmd_sync_schema_props_perms(ertype, syncprops=False, commit=commit)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1245
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1246
    # Workflows handling ######################################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1247
6805
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1248
    def cmd_make_workflowable(self, etype):
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1249
        """add workflow relations to an entity type to make it workflowable"""
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1250
        self.cmd_add_relation_definition(etype, 'in_state', 'State')
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1251
        self.cmd_add_relation_definition(etype, 'custom_workflow', 'Workflow')
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1252
        self.cmd_add_relation_definition('TrInfo', 'wf_info_for', etype)
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1253
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1254
    def cmd_add_workflow(self, name, wfof, default=True, commit=False,
6816
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1255
                         ensure_workflowable=True, **kwargs):
3582
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1256
        """
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1257
        create a new workflow and links it to entity types
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1258
         :type name: unicode
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1259
         :param name: name of the workflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1260
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1261
         :type wfof: string or list/tuple of strings
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1262
         :param wfof: entity type(s) having this workflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1263
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1264
         :type default: bool
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1265
         :param default: tells wether this is the default workflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1266
                   for the specified entity type(s); set it to false in
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1267
                   the case of a subworkflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1268
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1269
         :rtype: `Workflow`
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1270
        """
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1271
        wf = self.cmd_create_entity('Workflow', name=unicode(name),
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1272
                                    **kwargs)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1273
        if not isinstance(wfof, (list, tuple)):
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1274
            wfof = (wfof,)
6805
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1275
        def _missing_wf_rel(etype):
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1276
            return 'missing workflow relations, see make_workflowable(%s)' % etype
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1277
        for etype in wfof:
6805
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1278
            eschema = self.repo.schema[etype]
6816
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1279
            if ensure_workflowable:
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1280
                assert 'in_state' in eschema.subjrels, _missing_wf_rel(etype)
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1281
                assert 'custom_workflow' in eschema.subjrels, _missing_wf_rel(etype)
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1282
                assert 'wf_info_for' in eschema.objrels, _missing_wf_rel(etype)
2956
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1283
            rset = self.rqlexec(
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1284
                'SET X workflow_of ET WHERE X eid %(x)s, ET name %(et)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1285
                {'x': wf.eid, 'et': etype}, ask_confirm=False)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1286
            assert rset, 'unexistant entity type %s' % etype
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1287
            if default:
2956
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1288
                self.rqlexec(
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1289
                    'SET ET default_workflow X WHERE X eid %(x)s, ET name %(et)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1290
                    {'x': wf.eid, 'et': etype}, ask_confirm=False)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1291
        if commit:
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1292
            self.commit()
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1293
        return wf
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1294
7295
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1295
    def cmd_get_workflow_for(self, etype):
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1296
        """return default workflow for the given entity type"""
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1297
        rset = self.rqlexec('Workflow X WHERE ET default_workflow X, ET name %(et)s',
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1298
                            {'et': etype})
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1299
        return rset.get_entity(0, 0)
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1300
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1301
    # XXX remove once cmd_add_[state|transition] are removed
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1302
    def _get_or_create_wf(self, etypes):
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1303
        if not isinstance(etypes, (list, tuple)):
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1304
            etypes = (etypes,)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1305
        rset = self.rqlexec('Workflow X WHERE X workflow_of ET, ET name %(et)s',
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1306
                            {'et': etypes[0]})
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1307
        if rset:
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1308
            return rset.get_entity(0, 0)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1309
        return self.cmd_add_workflow('%s workflow' % ';'.join(etypes), etypes)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1310
4339
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1311
    @deprecated('[3.5] use add_workflow and Workflow.add_state method',
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1312
                stacklevel=3)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1313
    def cmd_add_state(self, name, stateof, initial=False, commit=False, **kwargs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1314
        """method to ease workflow definition: add a state for one or more
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1315
        entity type(s)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1316
        """
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1317
        wf = self._get_or_create_wf(stateof)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1318
        state = wf.add_state(name, initial, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1319
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1320
            self.commit()
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1321
        return state.eid
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1322
4339
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1323
    @deprecated('[3.5] use add_workflow and Workflow.add_transition method',
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1324
                stacklevel=3)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1325
    def cmd_add_transition(self, name, transitionof, fromstates, tostate,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1326
                           requiredgroups=(), conditions=(), commit=False, **kwargs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1327
        """method to ease workflow definition: add a transition for one or more
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1328
        entity type(s), from one or more state and to a single state
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1329
        """
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1330
        wf = self._get_or_create_wf(transitionof)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1331
        tr = wf.add_transition(name, fromstates, tostate, requiredgroups,
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1332
                               conditions, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1333
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1334
            self.commit()
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1335
        return tr.eid
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1336
4339
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1337
    @deprecated('[3.5] use Transition.set_transition_permissions method',
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1338
                stacklevel=3)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1339
    def cmd_set_transition_permissions(self, treid,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1340
                                       requiredgroups=(), conditions=(),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1341
                                       reset=True, commit=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1342
        """set or add (if `reset` is False) groups and conditions for a
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1343
        transition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1344
        """
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1345
        tr = self._cw.entity_from_eid(treid)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1346
        tr.set_transition_permissions(requiredgroups, conditions, reset)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1347
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1348
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1349
5556
9ab2b4c74baf [entity] introduce a new 'adapters' registry
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1350
    @deprecated('[3.5] use iworkflowable.fire_transition("transition") or '
9ab2b4c74baf [entity] introduce a new 'adapters' registry
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1351
                'iworkflowable.change_state("state")', stacklevel=3)
972
1efba3fd1364 set_state command in shell
sylvain.thenault@logilab.fr
parents: 934
diff changeset
  1352
    def cmd_set_state(self, eid, statename, commit=False):
5556
9ab2b4c74baf [entity] introduce a new 'adapters' registry
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1353
        self._cw.entity_from_eid(eid).cw_adapt_to('IWorkflowable').change_state(statename)
972
1efba3fd1364 set_state command in shell
sylvain.thenault@logilab.fr
parents: 934
diff changeset
  1354
        if commit:
1efba3fd1364 set_state command in shell
sylvain.thenault@logilab.fr
parents: 934
diff changeset
  1355
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1356
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1357
    # CWProperty handling ######################################################
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1358
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1359
    def cmd_property_value(self, pkey):
7111
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1360
        """retreive the site-wide persistent property value for the given key.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1361
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1362
        To get a user specific property value, use appropriate method on CWUser
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1363
        instance.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1364
        """
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1365
        rset = self.rqlexec(
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1366
            'Any V WHERE X is CWProperty, X pkey %(k)s, X value V, NOT X for_user U',
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1367
            {'k': pkey}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1368
        return rset[0][0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1369
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1370
    def cmd_set_property(self, pkey, value):
7111
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1371
        """set the site-wide persistent property value for the given key to the
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1372
        given value.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1373
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1374
        To set a user specific property value, use appropriate method on CWUser
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1375
        instance.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1376
        """
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1377
        value = unicode(value)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1378
        try:
7111
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1379
            prop = self.rqlexec(
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1380
                'CWProperty X WHERE X pkey %(k)s, NOT X for_user U',
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1381
                {'k': pkey}, ask_confirm=False).get_entity(0, 0)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
  1382
        except Exception:
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1383
            self.cmd_create_entity('CWProperty', pkey=unicode(pkey), value=value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1384
        else:
7111
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1385
            prop.set_attributes(value=value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1386
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1387
    # other data migration commands ###########################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1388
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1389
    @property
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1390
    def _cw(self):
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1391
        session = self.session
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1392
        if session is not None:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1393
            session.set_cnxset()
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1394
            return session
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1395
        return self.cnx.request()
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1396
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1397
    def cmd_storage_changed(self, etype, attribute):
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1398
        """migrate entities to a custom storage. The new storage is expected to
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1399
        be set, it will be temporarily removed for the migration.
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1400
        """
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1401
        from logilab.common.shellutils import ProgressBar
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1402
        source = self.repo.system_source
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1403
        storage = source.storage(etype, attribute)
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1404
        source.unset_storage(etype, attribute)
5552
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5551
diff changeset
  1405
        rset = self.rqlexec('Any X WHERE X is %s' % etype, ask_confirm=False)
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1406
        pb = ProgressBar(len(rset))
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1407
        for entity in rset.entities():
5551
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1408
            # fill cache. Do not fetch that attribute using the global rql query
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1409
            # since we may exhaust memory doing that....
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1410
            getattr(entity, attribute)
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1411
            storage.migrate_entity(entity, attribute)
5551
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1412
            # remove from entity cache to avoid memory exhaustion
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1413
            del entity[attribute]
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1414
            pb.update()
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1415
        print
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1416
        source.set_storage(etype, attribute, storage)
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1417
5132
260d73ad4f24 [cleaning] simpler implementation of cmd_create_entity
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5043
diff changeset
  1418
    def cmd_create_entity(self, etype, commit=False, **kwargs):
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1419
        """add a new entity of the given type"""
3708
95e8c3a9698a update prototype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3707
diff changeset
  1420
        entity = self._cw.create_entity(etype, **kwargs)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1421
        if commit:
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1422
            self.commit()
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1423
        return entity
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1424
7116
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1425
    def cmd_find_entities(self, etype, **kwargs):
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1426
        """find entities of the given type and attribute values"""
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1427
        return self._cw.find_entities(etype, **kwargs)
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1428
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1429
    def cmd_find_one_entity(self, etype, **kwargs):
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1430
        """find one entity of the given type and attribute values.
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1431
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1432
        raise :exc:`cubicweb.req.FindEntityError` if can not return one and only
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1433
        one entity.
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1434
        """
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1435
        return self._cw.find_one_entity(etype, **kwargs)
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1436
5928
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1437
    def cmd_update_etype_fti_weight(self, etype, weight):
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1438
        if self.repo.system_source.dbdriver == 'postgres':
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1439
            self.sqlexec('UPDATE appears SET weight=%(weight)s '
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1440
                         'FROM entities as X '
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1441
                         'WHERE X.eid=appears.uid AND X.type=%(type)s',
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1442
                         {'type': etype, 'weight': weight}, ask_confirm=False)
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1443
5850
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1444
    def cmd_reindex_entities(self, etypes=None):
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1445
        """force reindexaction of entities of the given types or of all
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1446
        indexable entity types
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1447
        """
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1448
        from cubicweb.server.checkintegrity import reindex_entities
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1449
        reindex_entities(self.repo.schema, self.session, etypes=etypes)
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1450
4367
fa02f7dccfe4 fix deprecation warnings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4344
diff changeset
  1451
    @deprecated('[3.5] use create_entity', stacklevel=3)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1452
    def cmd_add_entity(self, etype, *args, **kwargs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1453
        """add a new entity of the given type"""
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1454
        return self.cmd_create_entity(etype, *args, **kwargs).eid
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1455
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1456
    @contextmanager
7409
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1457
    def cmd_dropped_constraints(self, etype, attrname, cstrtype=None,
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1458
                                droprequired=False):
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1459
        """context manager to drop constraints temporarily on fs_schema
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1460
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1461
        `cstrtype` should be a constraint class (or a tuple of classes)
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1462
        and will be passed to isinstance directly
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1463
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1464
        For instance::
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1465
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1466
            >>> with dropped_constraints('MyType', 'myattr',
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1467
            ...                          UniqueConstraint, droprequired=True):
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1468
            ...     add_attribute('MyType', 'myattr')
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1469
            ...     # + instructions to fill MyType.myattr column
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1470
            ...
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1471
            >>>
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1472
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1473
        """
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1474
        rdef = self.fs_schema.eschema(etype).rdef(attrname)
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1475
        original_constraints = rdef.constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1476
        # remove constraints
7409
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1477
        if cstrtype:
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1478
            rdef.constraints = [cstr for cstr in original_constraints
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1479
                                if not (cstrtype and isinstance(cstr, cstrtype))]
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1480
        if droprequired:
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1481
            original_cardinality = rdef.cardinality
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1482
            rdef.cardinality = '?' + rdef.cardinality[1]
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1483
        yield
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1484
        # restore original constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1485
        rdef.constraints = original_constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1486
        if droprequired:
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1487
            rdef.cardinality = original_cardinality
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1488
        # update repository schema
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1489
        self.cmd_sync_schema_props_perms(rdef, syncperms=False)
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1490
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1491
    def sqlexec(self, sql, args=None, ask_confirm=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1492
        """execute the given sql if confirmed
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1493
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1494
        should only be used for low level stuff undoable with existing higher
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1495
        level actions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1496
        """
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1497
        if not ask_confirm or self.confirm('Execute sql: %s ?' % sql):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1498
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1499
                cu = self.session.system_sql(sql, args)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
  1500
            except Exception:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1501
                ex = sys.exc_info()[1]
6187
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6186
diff changeset
  1502
                if self.confirm('Error: %s\nabort?' % ex, pdb=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1503
                    raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1504
                return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1505
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1506
                return cu.fetchall()
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
  1507
            except Exception:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1508
                # no result to fetch
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1509
                return
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1510
4945
356662a6f06c [migration] new build_descr argument to rqlexec on the migration helper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4925
diff changeset
  1511
    def rqlexec(self, rql, kwargs=None, cachekey=None, build_descr=True,
6186
8ada7df95877 [migration] set ask_confirm=False by default on rqlexec
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6179
diff changeset
  1512
                ask_confirm=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1513
        """rql action"""
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1514
        if cachekey is not None:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1515
            warn('[3.8] cachekey is deprecated, you can safely remove this argument',
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1516
                 DeprecationWarning, stacklevel=2)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1517
        if not isinstance(rql, (tuple, list)):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1518
            rql = ( (rql, kwargs), )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1519
        res = None
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1520
        execute = self._cw.execute
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1521
        for rql, kwargs in rql:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1522
            if kwargs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1523
                msg = '%s (%s)' % (rql, kwargs)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1524
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1525
                msg = rql
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1526
            if not ask_confirm or self.confirm('Execute rql: %s ?' % msg):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1527
                try:
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1528
                    res = execute(rql, kwargs, build_descr=build_descr)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1529
                except Exception, ex:
6187
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6186
diff changeset
  1530
                    if self.confirm('Error: %s\nabort?' % ex, pdb=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1531
                        raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1532
        return res
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1533
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1534
    def rqliter(self, rql, kwargs=None, ask_confirm=True):
6686
070e3b23160d [migration] fix rqliter when some query arguments are given
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6582
diff changeset
  1535
        return ForRqlIterator(self, rql, kwargs, ask_confirm)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1536
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1537
    # broken db commands ######################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1538
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1539
    def cmd_change_attribute_type(self, etype, attr, newtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1540
        """low level method to change the type of an entity attribute. This is
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1541
        a quick hack which has some drawback:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1542
        * only works when the old type can be changed to the new type by the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1543
          underlying rdbms (eg using ALTER TABLE)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1544
        * the actual schema won't be updated until next startup
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1545
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1546
        rschema = self.repo.schema.rschema(attr)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1547
        oldtype = rschema.objects(etype)[0]
7928
c65fd337fd1d [migration] update old yams api in change_attribute_type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7915
diff changeset
  1548
        rdefeid = rschema.rdef(etype, oldtype).eid
7410
1dbf9a83bb49 [migractions] fix change_attribute_type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7409
diff changeset
  1549
        sql = ("UPDATE cw_CWAttribute "
1dbf9a83bb49 [migractions] fix change_attribute_type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7409
diff changeset
  1550
               "SET cw_to_entity=(SELECT cw_eid FROM cw_CWEType WHERE cw_name='%s')"
1dbf9a83bb49 [migractions] fix change_attribute_type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7409
diff changeset
  1551
               "WHERE cw_eid=%s") % (newtype, rdefeid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1552
        self.sqlexec(sql, ask_confirm=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1553
        dbhelper = self.repo.system_source.dbhelper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1554
        sqltype = dbhelper.TYPE_MAPPING[newtype]
7410
1dbf9a83bb49 [migractions] fix change_attribute_type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7409
diff changeset
  1555
        sql = 'ALTER TABLE cw_%s ALTER COLUMN cw_%s TYPE %s' % (etype, attr, sqltype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1556
        self.sqlexec(sql, ask_confirm=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1557
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1558
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1559
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1560
    def cmd_add_entity_type_table(self, etype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1561
        """low level method to create the sql table for an existing entity.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1562
        This may be useful on accidental desync between the repository schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1563
        and a sql database
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1564
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1565
        dbhelper = self.repo.system_source.dbhelper
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
  1566
        tablesql = eschema2sql(dbhelper, self.repo.schema.eschema(etype),
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
  1567
                               prefix=SQL_PREFIX)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1568
        for sql in tablesql.split(';'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1569
            if sql.strip():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1570
                self.sqlexec(sql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1571
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1572
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1573
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1574
    def cmd_add_relation_type_table(self, rtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1575
        """low level method to create the sql table for an existing relation.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1576
        This may be useful on accidental desync between the repository schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1577
        and a sql database
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1578
        """
7879
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
  1579
        tablesql = rschema2sql(self.repo.schema.rschema(rtype))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1580
        for sql in tablesql.split(';'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1581
            if sql.strip():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1582
                self.sqlexec(sql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1583
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1584
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1585
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
  1586
    @deprecated("[3.7] use session.disable_hook_categories('integrity')")
4834
b718626a0e60 move hooks activation control on session object, so we can have a per transaction control. Added a new `hooks_control` context manager for usual modification of hooks activation.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4763
diff changeset
  1587
    def cmd_deactivate_verification_hooks(self):
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
  1588
        self.session.disable_hook_categories('integrity')
4834
b718626a0e60 move hooks activation control on session object, so we can have a per transaction control. Added a new `hooks_control` context manager for usual modification of hooks activation.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4763
diff changeset
  1589
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
  1590
    @deprecated("[3.7] use session.enable_hook_categories('integrity')")
4834
b718626a0e60 move hooks activation control on session object, so we can have a per transaction control. Added a new `hooks_control` context manager for usual modification of hooks activation.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4763
diff changeset
  1591
    def cmd_reactivate_verification_hooks(self):
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
  1592
        self.session.enable_hook_categories('integrity')
4834
b718626a0e60 move hooks activation control on session object, so we can have a per transaction control. Added a new `hooks_control` context manager for usual modification of hooks activation.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4763
diff changeset
  1593
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1594
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1595
class ForRqlIterator:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1596
    """specific rql iterator to make the loop skipable"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1597
    def __init__(self, helper, rql, kwargs, ask_confirm):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1598
        self._h = helper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1599
        self.rql = rql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1600
        self.kwargs = kwargs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1601
        self.ask_confirm = ask_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1602
        self._rsetit = None
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1603
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1604
    def __iter__(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1605
        return self
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1606
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1607
    def _get_rset(self):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1608
        rql, kwargs = self.rql, self.kwargs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1609
        if kwargs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1610
            msg = '%s (%s)' % (rql, kwargs)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1611
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1612
            msg = rql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1613
        if self.ask_confirm:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1614
            if not self._h.confirm('Execute rql: %s ?' % msg):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1615
                raise StopIteration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1616
        try:
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1617
            return self._h._cw.execute(rql, kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1618
        except Exception, ex:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1619
            if self._h.confirm('Error: %s\nabort?' % ex):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1620
                raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1621
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1622
                raise StopIteration
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1623
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1624
    def next(self):
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1625
        if self._rsetit is not None:
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1626
            return self._rsetit.next()
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1627
        rset = self._get_rset()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1628
        self._rsetit = iter(rset)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1629
        return self._rsetit.next()
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1630
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1631
    def entities(self):
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1632
        try:
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1633
            rset = self._get_rset()
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1634
        except StopIteration:
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1635
            return []
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1636
        return rset.entities()