server/migractions.py
author Nicolas Chauvat <nicolas.chauvat@logilab.fr>
Tue, 09 Aug 2011 09:43:26 +0200
changeset 7896 4c954e1e73ef
parent 7894 ad0eeb0f7a8d
child 7920 5146c63e8e34
permissions -rw-r--r--
[lint] remove uses of "print >> sys.stderr" (closes #1908571)
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
7116
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
     1
# copyright 2003-2011 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
"""a class implementing basic actions used in migration scripts.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
The following schema actions are supported for now:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
* add/drop/rename attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
* add/drop entity/relation type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
* rename entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
The following data actions are supported for now:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
* add an entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
* execute raw RQL queries
5850
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
    28
"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    29
4925
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
    30
from __future__ import with_statement
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
    31
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    32
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    33
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    34
import sys
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    35
import os
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    36
import tarfile
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    37
import tempfile
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    38
import shutil
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    39
import os.path as osp
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 972
diff changeset
    40
from datetime import datetime
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
    41
from glob import glob
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
    42
from copy import copy
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
    43
from warnings import warn
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
    44
from contextlib import contextmanager
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    45
2613
5e19c2bb370e R [all] logilab.common 0.44 provides only deprecated
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2493
diff changeset
    46
from logilab.common.deprecation import deprecated
2107
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    47
from logilab.common.decorators import cached, clear_cache
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    48
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    49
from yams.constraints import SizeConstraint
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
from yams.schema2sql import eschema2sql, rschema2sql
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
    51
from yams.schema import RelationDefinitionSchema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
    53
from cubicweb import AuthenticationError, ExecutionError
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    54
from cubicweb.selectors import is_instance
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
    55
from cubicweb.schema import (ETYPE_NAME_MAP, META_RTYPES, VIRTUAL_RTYPES,
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
    56
                             PURE_VIRTUAL_RTYPES,
2926
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
    57
                             CubicWebRelationSchema, order_eschemas)
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    58
from cubicweb.cwvreg import CW_EVENT_MANAGER
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    59
from cubicweb.dbapi import get_repository, repo_connect
4021
280c910c8710 move i18n / migration modules from cw.common to cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4017
diff changeset
    60
from cubicweb.migration import MigrationHelper, yes
4925
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
    61
from cubicweb.server.session import hooks_control
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
    62
from cubicweb.server import hook
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
try:
2107
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    64
    from cubicweb.server import SOURCE_TYPES, schemaserial as ss
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
    65
    from cubicweb.server.utils import manager_userpasswd
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
    66
    from cubicweb.server.sqlutils import sqlexec, SQL_PREFIX
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    67
except ImportError: # LAX
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    68
    pass
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    69
6571
71b446d1cf93 [dependancy] do not use testlib from outside test: recent lgc versions recommends unittest2 while lgc.testlib needs it, and we don't want it to be necessary to run cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6323
diff changeset
    70
71b446d1cf93 [dependancy] do not use testlib from outside test: recent lgc versions recommends unittest2 while lgc.testlib needs it, and we don't want it to be necessary to run cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6323
diff changeset
    71
def mock_object(**params):
71b446d1cf93 [dependancy] do not use testlib from outside test: recent lgc versions recommends unittest2 while lgc.testlib needs it, and we don't want it to be necessary to run cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6323
diff changeset
    72
    return type('Mock', (), params)()
71b446d1cf93 [dependancy] do not use testlib from outside test: recent lgc versions recommends unittest2 while lgc.testlib needs it, and we don't want it to be necessary to run cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6323
diff changeset
    73
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    74
class ClearGroupMap(hook.Hook):
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    75
    __regid__ = 'cw.migration.clear_group_mapping'
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    76
    __select__ = hook.Hook.__select__ & is_instance('CWGroup')
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    77
    events = ('after_add_entity', 'after_update_entity',)
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    78
    def __call__(self):
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    79
        clear_cache(self.mih, 'group_mapping')
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    80
        self.mih._synchronized.clear()
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
    81
6205
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    82
    @classmethod
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    83
    def mih_register(cls, repo):
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    84
        # may be already registered in tests (e.g. unittest_migractions at
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    85
        # least)
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    86
        if not cls.__regid__ in repo.vreg['after_add_entity_hooks']:
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    87
            repo.vreg.register(ClearGroupMap)
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    88
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    89
class ServerMigrationHelper(MigrationHelper):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
    """specific migration helper for server side  migration scripts,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
    providind actions related to schema/data migration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    94
    def __init__(self, config, schema, interactive=True,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
                 repo=None, cnx=None, verbosity=1, connect=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
        MigrationHelper.__init__(self, config, interactive, verbosity)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    97
        if not interactive:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    98
            assert cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    99
            assert repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   100
        if cnx is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   101
            assert repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   102
            self._cnx = cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   103
            self.repo = repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   104
        elif connect:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   105
            self.repo_connect()
5019
72734c210836 [c-c] new server_maintenance hook, called on c-c shell / upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   106
        # no config on shell to a remote instance
5034
4781870e97d9 [maintainance] don't crash if we've no in-memory repository
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5019
diff changeset
   107
        if config is not None and (cnx or connect):
6203
d3dea5f84404 [test] during some tests (unittest_migrations), we should not register ClearGroupMap which is actually already there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6200
diff changeset
   108
            repo = self.repo
5019
72734c210836 [c-c] new server_maintenance hook, called on c-c shell / upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   109
            self.session.data['rebuild-infered'] = False
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   110
            # register a hook to clear our group_mapping cache and the
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   111
            # self._synchronized set when some group is added or updated
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   112
            ClearGroupMap.mih = self
6205
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
   113
            ClearGroupMap.mih_register(repo)
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
   114
            CW_EVENT_MANAGER.bind('after-registry-reload',
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
   115
                                  ClearGroupMap.mih_register, repo)
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   116
            # notify we're starting maintenance (called instead of server_start
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   117
            # which is called on regular start
6203
d3dea5f84404 [test] during some tests (unittest_migrations), we should not register ClearGroupMap which is actually already there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6200
diff changeset
   118
            repo.hm.call_hooks('server_maintenance', repo=repo)
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5034
diff changeset
   119
        if not schema and not getattr(config, 'quick_start', False):
7797
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   120
            insert_lperms = self.repo.get_versions()['cubicweb'] < (3, 14, 0) and 'localperms' in config.available_cubes()
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   121
            if insert_lperms:
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   122
                cubes = config._cubes
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   123
                config._cubes += ('localperms',)
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   124
            try:
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   125
                schema = config.load_schema(expand_cubes=True)
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   126
            finally:
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   127
                if insert_lperms:
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   128
                    config._cubes = cubes
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   129
        self.fs_schema = schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   130
        self._synchronized = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   131
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   132
    # overriden from base MigrationHelper ######################################
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   133
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   134
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
    def repo_connect(self):
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   136
        self.repo = get_repository(method='inmemory', config=self.config)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
        return self.repo
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   138
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   139
    def cube_upgraded(self, cube, version):
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   140
        self.cmd_set_property('system.version.%s' % cube.lower(),
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   141
                              unicode(version))
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   142
        self.commit()
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   143
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   144
    def shutdown(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   145
        if self.repo is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   146
            self.repo.shutdown()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   147
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   148
    def migrate(self, vcconf, toupgrade, options):
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   149
        if not options.fs_only:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   150
            if options.backup_db is None:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   151
                self.backup_database()
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   152
            elif options.backup_db:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   153
                self.backup_database(askconfirm=False)
4925
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
   154
        # disable notification during migration
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
   155
        with hooks_control(self.session, self.session.HOOKS_ALLOW_ALL, 'notification'):
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
   156
            super(ServerMigrationHelper, self).migrate(vcconf, toupgrade, options)
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   157
3715
e3ccadb126d7 [shell] make process_script available throuhg c-c shell / migration script context
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3708
diff changeset
   158
    def cmd_process_script(self, migrscript, funcname=None, *args, **kwargs):
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   159
        try:
6042
df9cafb8062c [migration] commit only sql script migration
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6035
diff changeset
   160
            return super(ServerMigrationHelper, self).cmd_process_script(
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   161
                  migrscript, funcname, *args, **kwargs)
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   162
        except ExecutionError, err:
7896
4c954e1e73ef [lint] remove uses of "print >> sys.stderr" (closes #1908571)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7894
diff changeset
   163
            sys.stderr.write("-> %s\n" % err)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   164
        except BaseException:
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   165
            self.rollback()
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   166
            raise
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   167
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   168
    # Adjust docstring
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   169
    cmd_process_script.__doc__ = MigrationHelper.cmd_process_script.__doc__
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   170
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   171
    # server specific migration methods ########################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   172
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   173
    def backup_database(self, backupfile=None, askconfirm=True, format='native'):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   174
        config = self.config
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2489
diff changeset
   175
        repo = self.repo_connect()
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   176
        # paths
3192
93c8fdcd943e windows doesn't like ':' in filesc
Aurelien Campeas
parents: 3120
diff changeset
   177
        timestamp = datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   178
        instbkdir = osp.join(config.appdatahome, 'backup')
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   179
        if not osp.exists(instbkdir):
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   180
            os.makedirs(instbkdir)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   181
        backupfile = backupfile or osp.join(instbkdir, '%s-%s.tar.gz'
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   182
                                            % (config.appid, timestamp))
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   183
        # check backup has to be done
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   184
        if osp.exists(backupfile) and not \
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   185
                self.confirm('Backup file %s exists, overwrite it?' % backupfile):
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   186
            print '-> no backup done.'
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   187
            return
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   188
        elif askconfirm and not self.confirm('Backup %s database?' % config.appid):
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   189
            print '-> no backup done.'
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   190
            return
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   191
        open(backupfile,'w').close() # kinda lock
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   192
        os.chmod(backupfile, 0600)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   193
        # backup
6918
9e607157d4cf [cw-ctl] use default `mkdtemp()` behaviour to store backup file internally
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6816
diff changeset
   194
        tmpdir = tempfile.mkdtemp()
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   195
        try:
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   196
            failed = False
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   197
            for source in repo.sources:
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   198
                try:
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   199
                    source.backup(osp.join(tmpdir, source.uri), self.confirm, format=format)
4893
15ae9a33a7f2 [db backup] fix name error in backup_to_file: we've to pass .confirm all along the chain as for restore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   200
                except Exception, ex:
15ae9a33a7f2 [db backup] fix name error in backup_to_file: we've to pass .confirm all along the chain as for restore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   201
                    print '-> error trying to backup %s [%s]' % (source.uri, ex)
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   202
                    if not self.confirm('Continue anyway?', default='n'):
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   203
                        raise SystemExit(1)
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   204
                    else:
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   205
                        failed = True
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   206
            with open(osp.join(tmpdir, 'format.txt'), 'w') as format_file:
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   207
                format_file.write('%s\n' % format)
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   208
            with open(osp.join(tmpdir, 'versions.txt'), 'w') as version_file:
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   209
                versions = repo.get_versions()
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   210
                for cube, version in versions.iteritems():
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   211
                    version_file.write('%s %s\n' % (cube, version))
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   212
            if not failed:
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   213
                bkup = tarfile.open(backupfile, 'w|gz')
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   214
                for filename in os.listdir(tmpdir):
4721
8f63691ccb7f pylint style fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   215
                    bkup.add(osp.join(tmpdir, filename), filename)
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   216
                bkup.close()
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   217
                # call hooks
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   218
                repo.hm.call_hooks('server_backup', repo=repo, timestamp=timestamp)
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   219
                # done
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   220
                print '-> backup file',  backupfile
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   221
        finally:
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   222
            shutil.rmtree(tmpdir)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   223
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2489
diff changeset
   224
    def restore_database(self, backupfile, drop=True, systemonly=True,
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   225
                         askconfirm=True, format='native'):
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   226
        # check
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   227
        if not osp.exists(backupfile):
6323
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   228
            raise ExecutionError("Backup file %s doesn't exist" % backupfile)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   229
        if askconfirm and not self.confirm('Restore %s database from %s ?'
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2926
diff changeset
   230
                                           % (self.config.appid, backupfile)):
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   231
            return
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   232
        # unpack backup
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   233
        tmpdir = tempfile.mkdtemp()
3105
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   234
        try:
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   235
            bkup = tarfile.open(backupfile, 'r|gz')
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   236
        except tarfile.ReadError:
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   237
            # assume restoring old backup
4344
066e7884e57d add source in backup/restore failure message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4339
diff changeset
   238
            shutil.copy(backupfile, osp.join(tmpdir, 'system'))
3105
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   239
        else:
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   240
            for name in bkup.getnames():
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   241
                if name[0] in '/.':
6323
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   242
                    raise ExecutionError('Security check failed, path starts with "/" or "."')
3105
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   243
            bkup.close() # XXX seek error if not close+open !?!
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   244
            bkup = tarfile.open(backupfile, 'r|gz')
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   245
            bkup.extractall(path=tmpdir)
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   246
            bkup.close()
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   247
        if osp.isfile(osp.join(tmpdir, 'format.txt')):
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   248
            with open(osp.join(tmpdir, 'format.txt')) as format_file:
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   249
                written_format = format_file.readline().strip()
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   250
                if written_format in ('portable', 'native'):
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   251
                    format = written_format
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   252
        self.config.init_cnxset_pool = False
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2926
diff changeset
   253
        repo = self.repo_connect()
2861
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   254
        for source in repo.sources:
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   255
            if systemonly and source.uri != 'system':
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   256
                continue
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   257
            try:
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   258
                source.restore(osp.join(tmpdir, source.uri), self.confirm, drop, format)
2861
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   259
            except Exception, exc:
4344
066e7884e57d add source in backup/restore failure message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4339
diff changeset
   260
                print '-> error trying to restore %s [%s]' % (source.uri, exc)
2861
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   261
                if not self.confirm('Continue anyway?', default='n'):
9cb3027407aa B offer to continue even when failed to restore systemonly
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   262
                    raise SystemExit(1)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   263
        shutil.rmtree(tmpdir)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   264
        # call hooks
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   265
        repo.init_cnxset_pool()
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   266
        repo.hm.call_hooks('server_restore', repo=repo, timestamp=backupfile)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   267
        print '-> database restored.'
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   268
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   269
    @property
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   270
    def cnx(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   271
        """lazy connection"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   272
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   273
            return self._cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   274
        except AttributeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   275
            sourcescfg = self.repo.config.sources()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   276
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   277
                login = sourcescfg['admin']['login']
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   278
                pwd = sourcescfg['admin']['password']
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   279
            except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   280
                login, pwd = manager_userpasswd()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   281
            while True:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   282
                try:
3659
993997b4b41d 3.6 update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3589
diff changeset
   283
                    self._cnx = repo_connect(self.repo, login, password=pwd)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   284
                    if not 'managers' in self._cnx.user(self.session).groups:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   285
                        print 'migration need an account in the managers group'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   286
                    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   287
                        break
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   288
                except AuthenticationError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   289
                    print 'wrong user/password'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
                except (KeyboardInterrupt, EOFError):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   291
                    print 'aborting...'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   292
                    sys.exit(0)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   293
                try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   294
                    login, pwd = manager_userpasswd()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   295
                except (KeyboardInterrupt, EOFError):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   296
                    print 'aborting...'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   297
                    sys.exit(0)
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   298
            self.session.keep_cnxset_mode('transaction')
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   299
            self.session.data['rebuild-infered'] = False
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   300
            return self._cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   301
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   302
    @property
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   303
    def session(self):
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   304
        if self.config is not None:
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   305
            session = self.repo._get_session(self.cnx.sessionid)
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   306
            if session.cnxset is None:
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   307
                session.set_read_security(False)
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   308
                session.set_write_security(False)
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   309
            session.set_cnxset()
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   310
            return session
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   311
        # no access to session on remote instance
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   312
        return None
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   313
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
    def commit(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   315
        if hasattr(self, '_cnx'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   316
            self._cnx.commit()
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   317
        if self.session:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   318
            self.session.set_cnxset()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   319
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   320
    def rollback(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
        if hasattr(self, '_cnx'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   322
            self._cnx.rollback()
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   323
        if self.session:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   324
            self.session.set_cnxset()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   325
6186
8ada7df95877 [migration] set ask_confirm=False by default on rqlexec
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6179
diff changeset
   326
    def rqlexecall(self, rqliter, ask_confirm=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   327
        for rql, kwargs in rqliter:
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   328
            self.rqlexec(rql, kwargs, ask_confirm=ask_confirm)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   330
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
    def _create_context(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   332
        """return a dictionary to use as migration script execution context"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   333
        context = super(ServerMigrationHelper, self)._create_context()
4017
a5b4d4f2a1c7 use commit in migration script instead of checkpoint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4011
diff changeset
   334
        context.update({'commit': self.checkpoint,
4330
cb7b68679501 make rollback available in shell (as commit is)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   335
                        'rollback': self.rollback,
4017
a5b4d4f2a1c7 use commit in migration script instead of checkpoint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4011
diff changeset
   336
                        'checkpoint': deprecated('[3.6] use commit')(self.checkpoint),
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   337
                        'sql': self.sqlexec,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   338
                        'rql': self.rqlexec,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   339
                        'rqliter': self.rqliter,
3707
78596919ede3 [c-c] fixes for shell w/ pyro instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3700
diff changeset
   340
                        'schema': self.repo.get_schema(),
78596919ede3 [c-c] fixes for shell w/ pyro instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3700
diff changeset
   341
                        'cnx': self.cnx,
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   342
                        'fsschema': self.fs_schema,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   343
                        'session' : self.session,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   344
                        'repo' : self.repo,
2788
8d3dbe577d3a R put version info in deprecation warnings
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   345
                        'synchronize_schema': deprecated()(self.cmd_sync_schema_props_perms), # 3.4
8d3dbe577d3a R put version info in deprecation warnings
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   346
                        'synchronize_eschema': deprecated()(self.cmd_sync_schema_props_perms), # 3.4
8d3dbe577d3a R put version info in deprecation warnings
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2759
diff changeset
   347
                        'synchronize_rschema': deprecated()(self.cmd_sync_schema_props_perms), # 3.4
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   348
                        })
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
        return context
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   350
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   351
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   352
    def group_mapping(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   353
        """cached group mapping"""
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   354
        return ss.group_mapping(self._cw)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   355
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   356
    @cached
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   357
    def cstrtype_mapping(self):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   358
        """cached constraint types mapping"""
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   359
        return ss.cstrtype_mapping(self._cw)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   360
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   361
    def exec_event_script(self, event, cubepath=None, funcname=None,
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   362
                          *args, **kwargs):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   363
        if cubepath:
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   364
            apc = osp.join(cubepath, 'migration', '%s.py' % event)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   365
        else:
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   366
            apc = osp.join(self.config.migration_scripts_dir(), '%s.py' % event)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   367
        if osp.exists(apc):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   368
            if self.config.free_wheel:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2788
diff changeset
   369
                self.cmd_deactivate_verification_hooks()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   370
            self.info('executing %s', apc)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   371
            confirm = self.confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   372
            execscript_confirm = self.execscript_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   373
            self.confirm = yes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   374
            self.execscript_confirm = yes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   375
            try:
3715
e3ccadb126d7 [shell] make process_script available throuhg c-c shell / migration script context
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3708
diff changeset
   376
                return self.cmd_process_script(apc, funcname, *args, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   377
            finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   378
                self.confirm = confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   379
                self.execscript_confirm = execscript_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   380
                if self.config.free_wheel:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2788
diff changeset
   381
                    self.cmd_reactivate_verification_hooks()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   382
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   383
    def install_custom_sql_scripts(self, directory, driver):
7479
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   384
        sql_scripts = []
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   385
        for fpath in glob(osp.join(directory, '*.sql.%s' % driver)):
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   386
            newname = osp.basename(fpath).replace('.sql.%s' % driver,
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   387
                                                  '.%s.sql' % driver)
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   388
            warn('[3.5.6] rename %s into %s' % (fpath, newname),
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   389
                 DeprecationWarning)
7479
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   390
            sql_scripts.append(fpath)
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   391
        sql_scripts += glob(osp.join(directory, '*.%s.sql' % driver))
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   392
        for fpath in sql_scripts:
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   393
            print '-> installing', fpath
7479
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   394
            try:
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   395
                sqlexec(open(fpath).read(), self.session.system_sql, False,
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   396
                        delimiter=';;')
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   397
            except Exception, exc:
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   398
                print '-> ERROR:', exc, ', skipping', fpath
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   399
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   400
    # schema synchronization internals ########################################
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   402
    def _synchronize_permissions(self, erschema, teid):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   403
        """permission synchronization for an entity or relation type"""
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   404
        assert teid, erschema
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   405
        if 'update' in erschema.ACTIONS or erschema.final:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   406
            # entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   407
            exprtype = u'ERQLExpression'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   408
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   409
            # relation type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   410
            exprtype = u'RRQLExpression'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   411
        gm = self.group_mapping()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   412
        confirm = self.verbosity >= 2
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   413
        # * remove possibly deprecated permission (eg in the persistent schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   414
        #   but not in the new schema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   415
        # * synchronize existing expressions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   416
        # * add new groups/expressions
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   417
        for action in erschema.ACTIONS:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   418
            perm = '%s_permission' % action
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   419
            # handle groups
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   420
            newgroups = list(erschema.get_groups(action))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
            for geid, gname in self.rqlexec('Any G, GN WHERE T %s G, G name GN, '
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   422
                                            'T eid %%(x)s' % perm, {'x': teid},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
                                            ask_confirm=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   424
                if not gname in newgroups:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   425
                    if not confirm or self.confirm('Remove %s permission of %s to %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   426
                                                   % (action, erschema, gname)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   427
                        self.rqlexec('DELETE T %s G WHERE G eid %%(x)s, T eid %s'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   428
                                     % (perm, teid),
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   429
                                     {'x': geid}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   430
                else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   431
                    newgroups.remove(gname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   432
            for gname in newgroups:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   433
                if not confirm or self.confirm('Grant %s permission of %s to %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   434
                                               % (action, erschema, gname)):
6075
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   435
                    try:
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   436
                        self.rqlexec('SET T %s G WHERE G eid %%(x)s, T eid %s'
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   437
                                     % (perm, teid),
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   438
                                     {'x': gm[gname]}, ask_confirm=False)
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   439
                    except KeyError:
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   440
                        self.error('can grant %s perm to unexistant group %s',
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   441
                                   action, gname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   442
            # handle rql expressions
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   443
            newexprs = dict((expr.expression, expr) for expr in erschema.get_rqlexprs(action))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   444
            for expreid, expression in self.rqlexec('Any E, EX WHERE T %s E, E expression EX, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   445
                                                    'T eid %s' % (perm, teid),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   446
                                                    ask_confirm=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   447
                if not expression in newexprs:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   448
                    if not confirm or self.confirm('Remove %s expression for %s permission of %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   449
                                                   % (expression, action, erschema)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   450
                        # deleting the relation will delete the expression entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   451
                        self.rqlexec('DELETE T %s E WHERE E eid %%(x)s, T eid %s'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   452
                                     % (perm, teid),
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   453
                                     {'x': expreid}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   454
                else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   455
                    newexprs.pop(expression)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   456
            for expression in newexprs.values():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   457
                expr = expression.expression
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   458
                if not confirm or self.confirm('Add %s expression for %s permission of %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   459
                                               % (expr, action, erschema)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   460
                    self.rqlexec('INSERT RQLExpression X: X exprtype %%(exprtype)s, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   461
                                 'X expression %%(expr)s, X mainvars %%(vars)s, T %s X '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   462
                                 'WHERE T eid %%(x)s' % perm,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   463
                                 {'expr': expr, 'exprtype': exprtype,
7161
e3f69df8dac7 fix failing tests before 3.12 release
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7116
diff changeset
   464
                                  'vars': u','.join(sorted(expression.mainvars)),
e3f69df8dac7 fix failing tests before 3.12 release
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7116
diff changeset
   465
                                  'x': teid},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   466
                                 ask_confirm=False)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   467
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   468
    def _synchronize_rschema(self, rtype, syncrdefs=True,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   469
                             syncperms=True, syncprops=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   470
        """synchronize properties of the persistent relation schema against its
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   471
        current definition:
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   472
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   473
        * description
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4367
diff changeset
   474
        * symmetric, meta
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   475
        * inlined
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   476
        * relation definitions if `syncrdefs`
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   477
        * permissions if `syncperms`
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   478
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   479
        physical schema changes should be handled by repository's schema hooks
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   480
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   481
        rtype = str(rtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   482
        if rtype in self._synchronized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   483
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   484
        self._synchronized.add(rtype)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   485
        rschema = self.fs_schema.rschema(rtype)
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   486
        reporschema = self.repo.schema.rschema(rtype)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   487
        if syncprops:
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   488
            assert reporschema.eid, reporschema
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   489
            self.rqlexecall(ss.updaterschema2rql(rschema, reporschema.eid),
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   490
                            ask_confirm=self.verbosity>=2)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   491
        if syncrdefs:
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   492
            for subj, obj in rschema.rdefs:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   493
                if (subj, obj) not in reporschema.rdefs:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   494
                    continue
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   495
                if rschema in VIRTUAL_RTYPES:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   496
                    continue
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   497
                self._synchronize_rdef_schema(subj, rschema, obj,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   498
                                              syncprops=syncprops,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   499
                                              syncperms=syncperms)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   500
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   501
    def _synchronize_eschema(self, etype, syncrdefs=True,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   502
                             syncperms=True, syncprops=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   503
        """synchronize properties of the persistent entity schema against
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   504
        its current definition:
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   505
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   506
        * description
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   507
        * internationalizable, fulltextindexed, indexed, meta
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   508
        * relations from/to this entity
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   509
        * __unique_together__
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   510
        * permissions if `syncperms`
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   511
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   512
        etype = str(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   513
        if etype in self._synchronized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   514
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   515
        self._synchronized.add(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   516
        repoeschema = self.repo.schema.eschema(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   517
        try:
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   518
            eschema = self.fs_schema.eschema(etype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   519
        except KeyError:
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   520
            return # XXX somewhat unexpected, no?...
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   521
        if syncprops:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   522
            repospschema = repoeschema.specializes()
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   523
            espschema = eschema.specializes()
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   524
            if repospschema and not espschema:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   525
                self.rqlexec('DELETE X specializes Y WHERE X is CWEType, X name %(x)s',
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   526
                             {'x': str(repoeschema)}, ask_confirm=False)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   527
            elif not repospschema and espschema:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   528
                self.rqlexec('SET X specializes Y WHERE X is CWEType, X name %(x)s, '
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   529
                             'Y is CWEType, Y name %(y)s',
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   530
                             {'x': str(repoeschema), 'y': str(espschema)},
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   531
                             ask_confirm=False)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   532
            self.rqlexecall(ss.updateeschema2rql(eschema, repoeschema.eid),
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   533
                            ask_confirm=self.verbosity >= 2)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   534
        if syncperms:
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   535
            self._synchronize_permissions(eschema, repoeschema.eid)
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   536
        if syncrdefs:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   537
            for rschema, targettypes, role in eschema.relation_definitions(True):
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   538
                if rschema in VIRTUAL_RTYPES:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   539
                    continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   540
                if role == 'subject':
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   541
                    if not rschema in repoeschema.subject_relations():
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   542
                        continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   543
                    subjtypes, objtypes = [etype], targettypes
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   544
                else: # role == 'object'
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   545
                    if not rschema in repoeschema.object_relations():
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   546
                        continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   547
                    subjtypes, objtypes = targettypes, [etype]
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   548
                self._synchronize_rschema(rschema, syncrdefs=False,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   549
                                          syncprops=syncprops, syncperms=syncperms)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   550
                reporschema = self.repo.schema.rschema(rschema)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   551
                for subj in subjtypes:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   552
                    for obj in objtypes:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   553
                        if (subj, obj) not in reporschema.rdefs:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   554
                            continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   555
                        self._synchronize_rdef_schema(subj, rschema, obj,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   556
                                                      syncprops=syncprops, syncperms=syncperms)
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   557
        if syncprops: # need to process __unique_together__ after rdefs were processed
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   558
            repo_unique_together = set([frozenset(ut)
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   559
                                        for ut in repoeschema._unique_together])
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   560
            unique_together = set([frozenset(ut)
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   561
                                   for ut in eschema._unique_together])
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   562
            for ut in repo_unique_together - unique_together:
6815
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   563
                restrictions = []
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   564
                substs = {'x': repoeschema.eid}
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   565
                for i, col in enumerate(ut):
6815
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   566
                    restrictions.append('C relations T%(i)d, '
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   567
                                       'T%(i)d name %%(T%(i)d)s' % {'i': i})
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   568
                    substs['T%d'%i] = col
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   569
                self.rqlexec('DELETE CWUniqueTogetherConstraint C '
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   570
                             'WHERE C constraint_of E, '
6815
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   571
                             '      E eid %%(x)s,'
7008
5d0be45ee69d cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6918
diff changeset
   572
                             '      %s' % ', '.join(restrictions),
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   573
                             substs)
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   574
            for ut in unique_together - repo_unique_together:
6815
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   575
                rql, substs = ss.uniquetogether2rql(eschema, ut)
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   576
                substs['x'] = repoeschema.eid
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6805
diff changeset
   577
                self.rqlexec(rql, substs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   578
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   579
    def _synchronize_rdef_schema(self, subjtype, rtype, objtype,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   580
                                 syncperms=True, syncprops=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   581
        """synchronize properties of the persistent relation definition schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   582
        against its current definition:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   583
        * order and other properties
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   584
        * constraints
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   585
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   586
        subjtype, objtype = str(subjtype), str(objtype)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   587
        rschema = self.fs_schema.rschema(rtype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   588
        reporschema = self.repo.schema.rschema(rschema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   589
        if (subjtype, rschema, objtype) in self._synchronized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   590
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   591
        self._synchronized.add((subjtype, rschema, objtype))
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4367
diff changeset
   592
        if rschema.symmetric:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   593
            self._synchronized.add((objtype, rschema, subjtype))
4949
f4dce73da26b [schema sync] don't try to synchronize infered relation defs, fixing a name error on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4947
diff changeset
   594
        rdef = rschema.rdef(subjtype, objtype)
f4dce73da26b [schema sync] don't try to synchronize infered relation defs, fixing a name error on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4947
diff changeset
   595
        if rdef.infered:
f4dce73da26b [schema sync] don't try to synchronize infered relation defs, fixing a name error on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4947
diff changeset
   596
            return # don't try to synchronize infered relation defs
4950
bca0873c0d6e [schema sync] fix another potential name error on synchronizing rdefs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4949
diff changeset
   597
        repordef = reporschema.rdef(subjtype, objtype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   598
        confirm = self.verbosity >= 2
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   599
        if syncprops:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   600
            # properties
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   601
            self.rqlexecall(ss.updaterdef2rql(rdef, repordef.eid),
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   602
                            ask_confirm=confirm)
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   603
            # constraints
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   604
            newconstraints = list(rdef.constraints)
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   605
            # 1. remove old constraints and update constraints of the same type
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   606
            # NOTE: don't use rschema.constraint_by_type because it may be
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   607
            #       out of sync with newconstraints when multiple
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   608
            #       constraints of the same type are used
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   609
            for cstr in repordef.constraints:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   610
                for newcstr in newconstraints:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   611
                    if newcstr.type() == cstr.type():
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   612
                        break
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   613
                else:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   614
                    newcstr = None
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   615
                if newcstr is None:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   616
                    self.rqlexec('DELETE X constrained_by C WHERE C eid %(x)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   617
                                 {'x': cstr.eid}, ask_confirm=confirm)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   618
                else:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   619
                    newconstraints.remove(newcstr)
4947
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   620
                    value = unicode(newcstr.serialize())
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   621
                    if value != unicode(cstr.serialize()):
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   622
                        self.rqlexec('SET X value %(v)s WHERE X eid %(x)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   623
                                     {'x': cstr.eid, 'v': value},
4947
c3ded0287295 don't update constraints when it's not necessary
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4904
diff changeset
   624
                                     ask_confirm=confirm)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   625
            # 2. add new constraints
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   626
            cstrtype_map = self.cstrtype_mapping()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   627
            self.rqlexecall(ss.constraints2rql(cstrtype_map, newconstraints,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   628
                                               repordef.eid),
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   629
                            ask_confirm=confirm)
4188
b3258d2afe04 fix virtual relation detection: erschema is no more a relation type but a relation def in _synchronize_permissions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4099
diff changeset
   630
        if syncperms and not rschema in VIRTUAL_RTYPES:
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   631
            self._synchronize_permissions(rdef, repordef.eid)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   632
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   633
    # base actions ############################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   634
4011
394f853bb653 [migration] write migration instructions for permissions handling on relation definition
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3998
diff changeset
   635
    def checkpoint(self, ask_confirm=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   636
        """checkpoint action"""
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   637
        if not ask_confirm or self.confirm('Commit now ?', shell=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   638
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   639
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   640
    def cmd_add_cube(self, cube, update_database=True):
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   641
        self.cmd_add_cubes( (cube,), update_database)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   642
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   643
    def cmd_add_cubes(self, cubes, update_database=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   644
        """update_database is telling if the database schema should be updated
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   645
        or if only the relevant eproperty should be inserted (for the case where
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2275
diff changeset
   646
        a cube has been extracted from an existing instance, so the
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   647
        cube schema is already in there)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   648
        """
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   649
        newcubes = super(ServerMigrationHelper, self).cmd_add_cubes(cubes)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   650
        if not newcubes:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   651
            return
2107
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   652
        for cube in newcubes:
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   653
            self.cmd_set_property('system.version.'+cube,
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   654
                                  self.config.cube_version(cube))
3836
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   655
            # ensure added cube is in config cubes
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   656
            # XXX worth restoring on error?
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   657
            if not cube in self.config._cubes:
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   658
                self.config._cubes += (cube,)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   659
        if not update_database:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   660
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   661
            return
1036
593df4919845 when reading the schema while adding/removing cubes, read schema in non-strict mode
sylvain.thenault@logilab.fr
parents: 1016
diff changeset
   662
        newcubes_schema = self.config.load_schema(construction_mode='non-strict')
3836
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   663
        # XXX we have to replace fs_schema, used in cmd_add_relation_type
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   664
        # etc. and fsschema of migration script contexts
6217
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6208
diff changeset
   665
        self.fs_schema = newcubes_schema
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6208
diff changeset
   666
        self.update_context('fsschema', self.fs_schema)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   667
        new = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   668
        # execute pre-create files
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   669
        driver = self.repo.system_source.dbdriver
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   670
        for pack in reversed(newcubes):
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   671
            cubedir = self.config.cube_dir(pack)
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   672
            self.install_custom_sql_scripts(osp.join(cubedir, 'schema'), driver)
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   673
            self.exec_event_script('precreate', cubedir)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   674
        # add new entity and relation types
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   675
        for rschema in newcubes_schema.relations():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   676
            if not rschema in self.repo.schema:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   677
                self.cmd_add_relation_type(rschema.type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   678
                new.add(rschema.type)
2926
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   679
        toadd = [eschema for eschema in newcubes_schema.entities()
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   680
                 if not eschema in self.repo.schema]
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   681
        for eschema in order_eschemas(toadd):
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   682
            self.cmd_add_entity_type(eschema.type)
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   683
            new.add(eschema.type)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   684
        # check if attributes has been added to existing entities
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   685
        for rschema in newcubes_schema.relations():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   686
            existingschema = self.repo.schema.rschema(rschema.type)
4099
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   687
            for (fromtype, totype) in rschema.rdefs:
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   688
                if (fromtype, totype) in existingschema.rdefs:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   689
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   690
                # check we should actually add the relation definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   691
                if not (fromtype in new or totype in new or rschema in new):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   692
                    continue
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   693
                self.cmd_add_relation_definition(str(fromtype), rschema.type,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   694
                                                 str(totype))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   695
        # execute post-create files
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   696
        for pack in reversed(newcubes):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   697
            self.exec_event_script('postcreate', self.config.cube_dir(pack))
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   698
            self.commit()
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   699
2124
5a0b02f37b23 set removedeps to False by default, raise an exception instead of a simple assertion for error, more remove_cube tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2122
diff changeset
   700
    def cmd_remove_cube(self, cube, removedeps=False):
2122
4ea13a828513 add removedeps option to remove_cube to control wether a cube's dependencies should be removed as well or not
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2107
diff changeset
   701
        removedcubes = super(ServerMigrationHelper, self).cmd_remove_cube(
4ea13a828513 add removedeps option to remove_cube to control wether a cube's dependencies should be removed as well or not
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2107
diff changeset
   702
            cube, removedeps)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   703
        if not removedcubes:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   704
            return
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   705
        fsschema = self.fs_schema
1036
593df4919845 when reading the schema while adding/removing cubes, read schema in non-strict mode
sylvain.thenault@logilab.fr
parents: 1016
diff changeset
   706
        removedcubes_schema = self.config.load_schema(construction_mode='non-strict')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   707
        reposchema = self.repo.schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   708
        # execute pre-remove files
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   709
        for pack in reversed(removedcubes):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   710
            self.exec_event_script('preremove', self.config.cube_dir(pack))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   711
        # remove cubes'entity and relation types
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   712
        for rschema in fsschema.relations():
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   713
            if not rschema in removedcubes_schema and rschema in reposchema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   714
                self.cmd_drop_relation_type(rschema.type)
2926
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   715
        toremove = [eschema for eschema in fsschema.entities()
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   716
                    if not eschema in removedcubes_schema
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   717
                    and eschema in reposchema]
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   718
        for eschema in reversed(order_eschemas(toremove)):
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   719
            self.cmd_drop_entity_type(eschema.type)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   720
        for rschema in fsschema.relations():
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   721
            if rschema in removedcubes_schema and rschema in reposchema:
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   722
                # check if attributes/relations has been added to entities from
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   723
                # other cubes
4099
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   724
                for fromtype, totype in rschema.rdefs:
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   725
                    if (fromtype, totype) not in removedcubes_schema[rschema.type].rdefs and \
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   726
                           (fromtype, totype) in reposchema[rschema.type].rdefs:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   727
                        self.cmd_drop_relation_definition(
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   728
                            str(fromtype), rschema.type, str(totype))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   729
        # execute post-remove files
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   730
        for cube in reversed(removedcubes):
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   731
            self.exec_event_script('postremove', self.config.cube_dir(cube))
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   732
            self.rqlexec('DELETE CWProperty X WHERE X pkey %(pk)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   733
                         {'pk': u'system.version.'+cube}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   734
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   735
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   736
    # schema migration actions ################################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   737
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   738
    def cmd_add_attribute(self, etype, attrname, attrtype=None, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   739
        """add a new attribute on the given entity type"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   740
        if attrtype is None:
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   741
            rschema = self.fs_schema.rschema(attrname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   742
            attrtype = rschema.objects(etype)[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   743
        self.cmd_add_relation_definition(etype, attrname, attrtype, commit=commit)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   744
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   745
    def cmd_drop_attribute(self, etype, attrname, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   746
        """drop an existing attribute from the given entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   747
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   748
        `attrname` is a string giving the name of the attribute to drop
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   749
        """
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   750
        try:
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   751
            rschema = self.repo.schema.rschema(attrname)
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   752
            attrtype = rschema.objects(etype)[0]
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   753
        except KeyError:
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   754
            print 'warning: attribute %s %s is not known, skip deletion' % (
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   755
                etype, attrname)
7558
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   756
        else:
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   757
            self.cmd_drop_relation_definition(etype, attrname, attrtype,
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   758
                                              commit=commit)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   759
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   760
    def cmd_rename_attribute(self, etype, oldname, newname, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   761
        """rename an existing attribute of the given entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   762
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   763
        `oldname` is a string giving the name of the existing attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   764
        `newname` is a string giving the name of the renamed attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   765
        """
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   766
        eschema = self.fs_schema.eschema(etype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   767
        attrtype = eschema.destination(newname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   768
        # have to commit this first step anyway to get the definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   769
        # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   770
        self.cmd_add_attribute(etype, newname, attrtype, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   771
        # skipp NULL values if the attribute is required
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   772
        rql = 'SET X %s VAL WHERE X is %s, X %s VAL' % (newname, etype, oldname)
4633
a85b4361fb22 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4580
diff changeset
   773
        card = eschema.rdef(newname).cardinality[0]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   774
        if card == '1':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   775
            rql += ', NOT X %s NULL' % oldname
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   776
        self.rqlexec(rql, ask_confirm=self.verbosity>=2)
3548
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   777
        # XXX if both attributes fulltext indexed, should skip fti rebuild
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   778
        # XXX if old attribute was fti indexed but not the new one old value
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   779
        # won't be removed from the index (this occurs on other kind of
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   780
        # fulltextindexed change...)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   781
        self.cmd_drop_attribute(etype, oldname, commit=commit)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   782
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   783
    def cmd_add_entity_type(self, etype, auto=True, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   784
        """register a new entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   785
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   786
        in auto mode, automatically register entity's relation where the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   787
        targeted type is known
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   788
        """
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   789
        instschema = self.repo.schema
7166
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7161
diff changeset
   790
        eschema = self.fs_schema.eschema(etype)
7558
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   791
        if etype in instschema and not (eschema.final and eschema.eid is None):
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   792
            print 'warning: %s already known, skip addition' % etype
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   793
            return
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   794
        confirm = self.verbosity >= 2
4043
39ae94e0c8b8 give group mapping where needed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4042
diff changeset
   795
        groupmap = self.group_mapping()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   796
        cstrtypemap = self.cstrtype_mapping()
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   797
        # register the entity into CWEType
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   798
        execute = self._cw.execute
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   799
        ss.execschemarql(execute, eschema, ss.eschema2rql(eschema, groupmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   800
        # add specializes relation if needed
5293
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   801
        specialized = eschema.specializes()
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   802
        if specialized:
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   803
            try:
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   804
                specialized.eid = instschema[specialized].eid
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   805
            except KeyError:
6323
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   806
                raise ExecutionError('trying to add entity type but parent type is '
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   807
                                     'not yet in the database schema')
5293
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   808
            self.rqlexecall(ss.eschemaspecialize2rql(eschema), ask_confirm=confirm)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   809
        # register entity's attributes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   810
        for rschema, attrschema in eschema.attribute_definitions():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   811
            # ignore those meta relations, they will be automatically added
2617
89c62b855f2e [R schema hooks] rename META_RELATIONS_TYPES into META_RTYPES, use it and other schema consts to avoid errors
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2616
diff changeset
   812
            if rschema.type in META_RTYPES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   813
                continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   814
            if not rschema.type in instschema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   815
                # need to add the relation type and to commit to get it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   816
                # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   817
                self.cmd_add_relation_type(rschema.type, False, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   818
            # register relation definition
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   819
            rdef = self._get_rdef(rschema, eschema, eschema.destination(rschema))
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   820
            ss.execschemarql(execute, rdef, ss.rdef2rql(rdef, cstrtypemap, groupmap),)
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   821
        # take care to newly introduced base class
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   822
        # XXX some part of this should probably be under the "if auto" block
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   823
        for spschema in eschema.specialized_by(recursive=False):
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   824
            try:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   825
                instspschema = instschema[spschema]
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   826
            except KeyError:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   827
                # specialized entity type not in schema, ignore
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   828
                continue
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   829
            if instspschema.specializes() != eschema:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   830
                self.rqlexec('SET D specializes P WHERE D eid %(d)s, P name %(pn)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   831
                             {'d': instspschema.eid, 'pn': eschema.type},
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   832
                             ask_confirm=confirm)
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   833
                for rschema, tschemas, role in spschema.relation_definitions(True):
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   834
                    for tschema in tschemas:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   835
                        if not tschema in instschema:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   836
                            continue
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   837
                        if role == 'subject':
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   838
                            subjschema = spschema
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   839
                            objschema = tschema
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3582
diff changeset
   840
                            if rschema.final and rschema in instspschema.subjrels:
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   841
                                # attribute already set, has_rdef would check if
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   842
                                # it's of the same type, we don't want this so
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   843
                                # simply skip here
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   844
                                continue
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   845
                        elif role == 'object':
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   846
                            subjschema = tschema
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   847
                            objschema = spschema
4633
a85b4361fb22 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4580
diff changeset
   848
                        if (rschema.rdef(subjschema, objschema).infered
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   849
                            or (instschema.has_relation(rschema) and
4633
a85b4361fb22 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4580
diff changeset
   850
                                (subjschema, objschema) in instschema[rschema].rdefs)):
4721
8f63691ccb7f pylint style fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   851
                            continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   852
                        self.cmd_add_relation_definition(
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   853
                            subjschema.type, rschema.type, objschema.type)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   854
        if auto:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   855
            # we have commit here to get relation types actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   856
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   857
            added = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   858
            for rschema in eschema.subject_relations():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   859
                # attribute relation have already been processed and
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   860
                # 'owned_by'/'created_by' will be automatically added
2617
89c62b855f2e [R schema hooks] rename META_RELATIONS_TYPES into META_RTYPES, use it and other schema consts to avoid errors
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2616
diff changeset
   861
                if rschema.final or rschema.type in META_RTYPES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   862
                    continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   863
                rtypeadded = rschema.type in instschema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   864
                for targetschema in rschema.objects(etype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   865
                    # ignore relations where the targeted type is not in the
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2275
diff changeset
   866
                    # current instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   867
                    targettype = targetschema.type
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   868
                    if not targettype in instschema and targettype != etype:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   869
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   870
                    if not rtypeadded:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   871
                        # need to add the relation type and to commit to get it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   872
                        # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   873
                        added.append(rschema.type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   874
                        self.cmd_add_relation_type(rschema.type, False, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   875
                        rtypeadded = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   876
                    # register relation definition
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4367
diff changeset
   877
                    # remember this two avoid adding twice non symmetric relation
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   878
                    # such as "Emailthread forked_from Emailthread"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   879
                    added.append((etype, rschema.type, targettype))
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   880
                    rdef = self._get_rdef(rschema, eschema, targetschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   881
                    ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   882
                                     ss.rdef2rql(rdef, cstrtypemap, groupmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   883
            for rschema in eschema.object_relations():
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   884
                if rschema.type in META_RTYPES:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   885
                    continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   886
                rtypeadded = rschema.type in instschema or rschema.type in added
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   887
                for targetschema in rschema.subjects(etype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   888
                    # ignore relations where the targeted type is not in the
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2275
diff changeset
   889
                    # current instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   890
                    targettype = targetschema.type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   891
                    # don't check targettype != etype since in this case the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   892
                    # relation has already been added as a subject relation
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   893
                    if not targettype in instschema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   894
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   895
                    if not rtypeadded:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   896
                        # need to add the relation type and to commit to get it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   897
                        # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   898
                        self.cmd_add_relation_type(rschema.type, False, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   899
                        rtypeadded = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   900
                    elif (targettype, rschema.type, etype) in added:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   901
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   902
                    # register relation definition
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   903
                    rdef = self._get_rdef(rschema, targetschema, eschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   904
                    ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   905
                                     ss.rdef2rql(rdef, cstrtypemap, groupmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   906
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   907
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   908
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   909
    def cmd_drop_entity_type(self, etype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   910
        """unregister an existing entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   911
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   912
        This will trigger deletion of necessary relation types and definitions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   913
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   914
        # XXX what if we delete an entity type which is specialized by other types
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   915
        # unregister the entity from CWEType
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   916
        self.rqlexec('DELETE CWEType X WHERE X name %(etype)s', {'etype': etype},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   917
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   918
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   919
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   920
6115
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   921
    def cmd_rename_entity_type(self, oldname, newname, attrs=None, commit=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   922
        """rename an existing entity type in the persistent schema
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   923
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   924
        `oldname` is a string giving the name of the existing entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   925
        `newname` is a string giving the name of the renamed entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   926
        """
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   927
        schema = self.repo.schema
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   928
        if newname in schema:
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   929
            assert oldname in ETYPE_NAME_MAP, \
6115
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   930
                   '%s should be mapped to %s in ETYPE_NAME_MAP' % (oldname,
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   931
                                                                    newname)
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   932
            if attrs is None:
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   933
                attrs = ','.join(SQL_PREFIX + rschema.type
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   934
                                 for rschema in schema[newname].subject_relations()
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   935
                                 if (rschema.final or rschema.inlined)
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   936
                                 and not rschema in PURE_VIRTUAL_RTYPES)
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   937
            else:
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   938
                attrs += ('eid', 'creation_date', 'modification_date', 'cwuri')
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   939
                attrs = ','.join(SQL_PREFIX + attr for attr in attrs)
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   940
            self.sqlexec('INSERT INTO %s%s(%s) SELECT %s FROM %s%s' % (
5999
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   941
                SQL_PREFIX, newname, attrs, attrs, SQL_PREFIX, oldname),
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   942
                         ask_confirm=False)
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   943
            # old entity type has not been added to the schema, can't gather it
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   944
            new = schema.eschema(newname)
6117
f3fdfcbda54b cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6116
diff changeset
   945
            oldeid = self.rqlexec('CWEType ET WHERE ET name %(on)s',
f3fdfcbda54b cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6116
diff changeset
   946
                                  {'on': oldname}, ask_confirm=False)[0][0]
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   947
            # backport old type relations to new type
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   948
            # XXX workflows, other relations?
5999
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   949
            for r1, rr1 in [('from_entity', 'to_entity'),
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   950
                            ('to_entity', 'from_entity')]:
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   951
                self.rqlexec('SET X %(r1)s NET WHERE X %(r1)s OET, '
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   952
                             'NOT EXISTS(X2 %(r1)s NET, X relation_type XRT, '
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   953
                             'X2 relation_type XRT, X %(rr1)s XTE, X2 %(rr1)s XTE), '
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   954
                             'OET eid %%(o)s, NET eid %%(n)s' % locals(),
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   955
                             {'o': oldeid, 'n': new.eid}, ask_confirm=False)
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   956
            # backport is / is_instance_of relation to new type
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   957
            for rtype in ('is', 'is_instance_of'):
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   958
                self.sqlexec('UPDATE %s_relation SET eid_to=%s WHERE eid_to=%s'
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   959
                             % (rtype, new.eid, oldeid), ask_confirm=False)
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   960
            # delete relations using SQL to avoid relations content removal
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   961
            # triggered by schema synchronization hooks.
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   962
            session = self.session
6116
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   963
            for rdeftype in ('CWRelation', 'CWAttribute'):
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   964
                thispending = set( (eid for eid, in self.sqlexec(
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   965
                    'SELECT cw_eid FROM cw_%s WHERE cw_from_entity=%%(eid)s OR '
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   966
                    ' cw_to_entity=%%(eid)s' % rdeftype,
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   967
                    {'eid': oldeid}, ask_confirm=False)) )
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   968
                # we should add deleted eids into pending eids else we may
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   969
                # get some validation error on commit since integrity hooks
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   970
                # may think some required relation is missing... This also ensure
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   971
                # repository caches are properly cleanup
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   972
                hook.CleanupDeletedEidsCacheOp.get_instance(session).union(thispending)
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   973
                # and don't forget to remove record from system tables
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   974
                entities = [session.entity_from_eid(eid, rdeftype) for eid in thispending]
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   975
                self.repo.system_source.delete_info_multi(session, entities, 'system')
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   976
                self.sqlexec('DELETE FROM cw_%s WHERE cw_from_entity=%%(eid)s OR '
6116
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   977
                             'cw_to_entity=%%(eid)s' % rdeftype,
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   978
                             {'eid': oldeid}, ask_confirm=False)
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   979
                # now we have to manually cleanup relations pointing to deleted
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   980
                # entities
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   981
                thiseids = ','.join(str(eid) for eid in thispending)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   982
                for rschema, ttypes, role in schema[rdeftype].relation_definitions():
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   983
                    if rschema.type in VIRTUAL_RTYPES:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   984
                        continue
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   985
                    sqls = []
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   986
                    if role == 'object':
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   987
                        if rschema.inlined:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   988
                            for eschema in ttypes:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   989
                                sqls.append('DELETE FROM cw_%s WHERE cw_%s IN(%%s)'
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   990
                                            % (eschema, rschema))
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   991
                        else:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   992
                            sqls.append('DELETE FROM %s_relation WHERE eid_to IN(%%s)'
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   993
                                        % rschema)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   994
                    elif not rschema.inlined:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   995
                        sqls.append('DELETE FROM %s_relation WHERE eid_from IN(%%s)'
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   996
                                    % rschema)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   997
                    for sql in sqls:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   998
                        self.sqlexec(sql % thiseids, ask_confirm=False)
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   999
            # remove the old type: use rql to propagate deletion
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
  1000
            self.rqlexec('DELETE CWEType ET WHERE ET name %(on)s', {'on': oldname},
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
  1001
                         ask_confirm=False)
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1002
        else:
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
  1003
            self.rqlexec('SET ET name %(newname)s WHERE ET is CWEType, ET name %(on)s',
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
  1004
                         {'newname' : unicode(newname), 'on' : oldname},
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1005
                         ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1006
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1007
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1008
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1009
    def cmd_add_relation_type(self, rtype, addrdef=True, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1010
        """register a new relation type named `rtype`, as described in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1011
        schema description file.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1012
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1013
        `addrdef` is a boolean value; when True, it will also add all relations
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1014
        of the type just added found in the schema definition file. Note that it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1015
        implies an intermediate "commit" which commits the relation type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1016
        creation (but not the relation definitions themselves, for which
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1017
        committing depends on the `commit` argument value).
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1018
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1019
        """
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1020
        reposchema = self.repo.schema
7558
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
  1021
        if rtype in reposchema:
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1022
            print 'warning: relation type %s is already known, skip addition' % (
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1023
                rtype)
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1024
            return
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
  1025
        rschema = self.fs_schema.rschema(rtype)
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1026
        execute = self._cw.execute
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1027
        # register the relation into CWRType and insert necessary relation
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1028
        # definitions
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1029
        ss.execschemarql(execute, rschema, ss.rschema2rql(rschema, addrdef=False))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1030
        if addrdef:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1031
            self.commit()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1032
            gmap = self.group_mapping()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1033
            cmap = self.cstrtype_mapping()
6073
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1034
            done = set()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1035
            for rdef in rschema.rdefs.itervalues():
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1036
                if not (reposchema.has_entity(rdef.subject)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1037
                        and reposchema.has_entity(rdef.object)):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1038
                    continue
6073
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1039
                # symmetric relations appears twice
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1040
                if (rdef.subject, rdef.object) in done:
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1041
                    continue
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1042
                done.add( (rdef.subject, rdef.object) )
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1043
                self._set_rdef_eid(rdef)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1044
                ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1045
                                 ss.rdef2rql(rdef, cmap, gmap))
2700
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1046
            if rtype in META_RTYPES:
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1047
                # if the relation is in META_RTYPES, ensure we're adding it for
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1048
                # all entity types *in the persistent schema*, not only those in
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1049
                # the fs schema
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1050
                for etype in self.repo.schema.entities():
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1051
                    if not etype in self.fs_schema:
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1052
                        # get sample object type and rproperties
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1053
                        objtypes = rschema.objects()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1054
                        assert len(objtypes) == 1, objtypes
2700
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1055
                        objtype = objtypes[0]
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1056
                        rdef = copy(rschema.rdef(rschema.subjects(objtype)[0], objtype))
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1057
                        rdef.subject = etype
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1058
                        rdef.rtype = self.repo.schema.rschema(rschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1059
                        rdef.object = self.repo.schema.rschema(objtype)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1060
                        ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1061
                                         ss.rdef2rql(rdef, cmap, gmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1062
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1063
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1064
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1065
    def cmd_drop_relation_type(self, rtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1066
        """unregister an existing relation type"""
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1067
        # unregister the relation from CWRType
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1068
        self.rqlexec('DELETE CWRType X WHERE X name %r' % rtype,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1069
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1070
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1071
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1072
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1073
    def cmd_rename_relation(self, oldname, newname, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1074
        """rename an existing relation
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1075
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1076
        `oldname` is a string giving the name of the existing relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1077
        `newname` is a string giving the name of the renamed relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1078
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1079
        self.cmd_add_relation_type(newname, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1080
        self.rqlexec('SET X %s Y WHERE X %s Y' % (newname, oldname),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1081
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1082
        self.cmd_drop_relation_type(oldname, commit=commit)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1083
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1084
    def cmd_add_relation_definition(self, subjtype, rtype, objtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1085
        """register a new relation definition, from its definition found in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1086
        schema definition file
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1087
        """
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
  1088
        rschema = self.fs_schema.rschema(rtype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1089
        if not rtype in self.repo.schema:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1090
            self.cmd_add_relation_type(rtype, addrdef=False, commit=True)
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1091
        if (subjtype, objtype) in self.repo.schema.rschema(rtype).rdefs:
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1092
            print 'warning: relation %s %s %s is already known, skip addition' % (
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1093
                subjtype, rtype, objtype)
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1094
            return
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1095
        execute = self._cw.execute
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1096
        rdef = self._get_rdef(rschema, subjtype, objtype)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1097
        ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1098
                         ss.rdef2rql(rdef, self.cstrtype_mapping(),
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1099
                                     self.group_mapping()))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1100
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1101
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1102
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1103
    def _get_rdef(self, rschema, subjtype, objtype):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1104
        return self._set_rdef_eid(rschema.rdefs[(subjtype, objtype)])
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1105
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1106
    def _set_rdef_eid(self, rdef):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1107
        for attr in ('rtype', 'subject', 'object'):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1108
            schemaobj = getattr(rdef, attr)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1109
            if getattr(schemaobj, 'eid', None) is None:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1110
                schemaobj.eid =  self.repo.schema[schemaobj].eid
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1111
                assert schemaobj.eid is not None
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1112
        return rdef
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1113
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1114
    def cmd_drop_relation_definition(self, subjtype, rtype, objtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1115
        """unregister an existing relation definition"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1116
        rschema = self.repo.schema.rschema(rtype)
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1117
        # unregister the definition from CWAttribute or CWRelation
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3582
diff changeset
  1118
        if rschema.final:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1119
            etype = 'CWAttribute'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1120
        else:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1121
            etype = 'CWRelation'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1122
        rql = ('DELETE %s X WHERE X from_entity FE, FE name "%s",'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1123
               'X relation_type RT, RT name "%s", X to_entity TE, TE name "%s"')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1124
        self.rqlexec(rql % (etype, subjtype, rtype, objtype),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1125
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1126
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1127
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1128
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1129
    def cmd_sync_schema_props_perms(self, ertype=None, syncperms=True,
1409
f4dee84a618f sql attributes bugfix
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 1399
diff changeset
  1130
                                    syncprops=True, syncrdefs=True, commit=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1131
        """synchronize the persistent schema against the current definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1132
        schema.
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1133
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1134
        `ertype` can be :
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1135
        - None, in that case everything will be synced ;
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1136
        - a string, it should be an entity type or
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1137
          a relation type. In that case, only the corresponding
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1138
          entities / relations will be synced ;
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1139
        - an rdef object to synchronize only this specific relation definition
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1140
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1141
        It will synch common stuff between the definition schema and the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1142
        actual persistent schema, it won't add/remove any entity or relation.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1143
        """
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1144
        assert syncperms or syncprops, 'nothing to do'
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1145
        if ertype is not None:
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1146
            if isinstance(ertype, RelationDefinitionSchema):
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1147
                ertype = ertype.as_triple()
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1148
            if isinstance(ertype, (tuple, list)):
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1149
                assert len(ertype) == 3, 'not a relation definition'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
  1150
                self._synchronize_rdef_schema(ertype[0], ertype[1], ertype[2],
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1151
                                              syncperms=syncperms,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1152
                                              syncprops=syncprops)
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1153
            else:
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1154
                erschema = self.repo.schema[ertype]
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1155
                if isinstance(erschema, CubicWebRelationSchema):
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1156
                    self._synchronize_rschema(erschema, syncrdefs=syncrdefs,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1157
                                              syncperms=syncperms,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1158
                                              syncprops=syncprops)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1159
                else:
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1160
                    self._synchronize_eschema(erschema, syncrdefs=syncrdefs,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1161
                                              syncperms=syncperms,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1162
                                              syncprops=syncprops)
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1163
        else:
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1164
            for etype in self.repo.schema.entities():
7505
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1165
                if etype.eid is None:
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1166
                     # not yet added final etype (thing to BigInt defined in
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1167
                     # yams though 3.13 migration not done yet)
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1168
                    continue
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1169
                self._synchronize_eschema(etype, syncrdefs=syncrdefs,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1170
                                          syncprops=syncprops, syncperms=syncperms)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1171
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1172
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1173
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1174
    def cmd_change_relation_props(self, subjtype, rtype, objtype,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1175
                                  commit=True, **kwargs):
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1176
        """change some properties of a relation definition
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1177
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1178
        you usually want to use sync_schema_props_perms instead.
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1179
        """
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1180
        assert kwargs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1181
        restriction = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1182
        if subjtype and subjtype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1183
            restriction.append('X from_entity FE, FE name "%s"' % subjtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1184
        if objtype and objtype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1185
            restriction.append('X to_entity TE, TE name "%s"' % objtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1186
        if rtype and rtype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1187
            restriction.append('X relation_type RT, RT name "%s"' % rtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1188
        assert restriction
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1189
        values = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1190
        for k, v in kwargs.items():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1191
            values.append('X %s %%(%s)s' % (k, k))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1192
            if isinstance(v, str):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1193
                kwargs[k] = unicode(v)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1194
        rql = 'SET %s WHERE %s' % (','.join(values), ','.join(restriction))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1195
        self.rqlexec(rql, kwargs, ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1196
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1197
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1198
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1199
    def cmd_set_size_constraint(self, etype, rtype, size, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1200
        """set change size constraint of a string attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1201
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1202
        if size is None any size constraint will be removed.
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1203
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1204
        you usually want to use sync_schema_props_perms instead.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1205
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1206
        oldvalue = None
4695
4aaf87e7f79e 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4650
diff changeset
  1207
        for constr in self.repo.schema.eschema(etype).rdef(rtype).constraints:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1208
            if isinstance(constr, SizeConstraint):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1209
                oldvalue = constr.max
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1210
        if oldvalue == size:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1211
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1212
        if oldvalue is None and not size is None:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1213
            ceid = self.rqlexec('INSERT CWConstraint C: C value %(v)s, C cstrtype CT '
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1214
                                'WHERE CT name "SizeConstraint"',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1215
                                {'v': SizeConstraint(size).serialize()},
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1216
                                ask_confirm=self.verbosity>=2)[0][0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1217
            self.rqlexec('SET X constrained_by C WHERE X from_entity S, X relation_type R, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1218
                         'S name "%s", R name "%s", C eid %s' % (etype, rtype, ceid),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1219
                         ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1220
        elif not oldvalue is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1221
            if not size is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1222
                self.rqlexec('SET C value %%(v)s WHERE X from_entity S, X relation_type R,'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1223
                             'X constrained_by C, C cstrtype CT, CT name "SizeConstraint",'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1224
                             'S name "%s", R name "%s"' % (etype, rtype),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1225
                             {'v': unicode(SizeConstraint(size).serialize())},
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1226
                             ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1227
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1228
                self.rqlexec('DELETE X constrained_by C WHERE X from_entity S, X relation_type R,'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1229
                             'X constrained_by C, C cstrtype CT, CT name "SizeConstraint",'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1230
                             'S name "%s", R name "%s"' % (etype, rtype),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1231
                             ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1232
                # cleanup unused constraints
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1233
                self.rqlexec('DELETE CWConstraint C WHERE NOT X constrained_by C')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1234
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1235
            self.commit()
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1236
3577
067610a0e12f add version number to deprecation messages
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3548
diff changeset
  1237
    @deprecated('[3.2] use sync_schema_props_perms(ertype, syncprops=False)')
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1238
    def cmd_synchronize_permissions(self, ertype, commit=True):
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1239
        self.cmd_sync_schema_props_perms(ertype, syncprops=False, commit=commit)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1240
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1241
    # Workflows handling ######################################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1242
6805
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1243
    def cmd_make_workflowable(self, etype):
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1244
        """add workflow relations to an entity type to make it workflowable"""
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1245
        self.cmd_add_relation_definition(etype, 'in_state', 'State')
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1246
        self.cmd_add_relation_definition(etype, 'custom_workflow', 'Workflow')
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1247
        self.cmd_add_relation_definition('TrInfo', 'wf_info_for', etype)
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1248
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1249
    def cmd_add_workflow(self, name, wfof, default=True, commit=False,
6816
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1250
                         ensure_workflowable=True, **kwargs):
3582
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1251
        """
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1252
        create a new workflow and links it to entity types
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1253
         :type name: unicode
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1254
         :param name: name of the workflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1255
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1256
         :type wfof: string or list/tuple of strings
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1257
         :param wfof: entity type(s) having this workflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1258
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1259
         :type default: bool
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1260
         :param default: tells wether this is the default workflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1261
                   for the specified entity type(s); set it to false in
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1262
                   the case of a subworkflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1263
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1264
         :rtype: `Workflow`
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1265
        """
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1266
        wf = self.cmd_create_entity('Workflow', name=unicode(name),
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1267
                                    **kwargs)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1268
        if not isinstance(wfof, (list, tuple)):
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1269
            wfof = (wfof,)
6805
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1270
        def _missing_wf_rel(etype):
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1271
            return 'missing workflow relations, see make_workflowable(%s)' % etype
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1272
        for etype in wfof:
6805
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1273
            eschema = self.repo.schema[etype]
6816
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1274
            if ensure_workflowable:
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1275
                assert 'in_state' in eschema.subjrels, _missing_wf_rel(etype)
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1276
                assert 'custom_workflow' in eschema.subjrels, _missing_wf_rel(etype)
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1277
                assert 'wf_info_for' in eschema.objrels, _missing_wf_rel(etype)
2956
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1278
            rset = self.rqlexec(
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1279
                'SET X workflow_of ET WHERE X eid %(x)s, ET name %(et)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1280
                {'x': wf.eid, 'et': etype}, ask_confirm=False)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1281
            assert rset, 'unexistant entity type %s' % etype
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1282
            if default:
2956
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1283
                self.rqlexec(
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1284
                    'SET ET default_workflow X WHERE X eid %(x)s, ET name %(et)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1285
                    {'x': wf.eid, 'et': etype}, ask_confirm=False)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1286
        if commit:
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1287
            self.commit()
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1288
        return wf
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1289
7295
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1290
    def cmd_get_workflow_for(self, etype):
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1291
        """return default workflow for the given entity type"""
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1292
        rset = self.rqlexec('Workflow X WHERE ET default_workflow X, ET name %(et)s',
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1293
                            {'et': etype})
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1294
        return rset.get_entity(0, 0)
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1295
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1296
    # XXX remove once cmd_add_[state|transition] are removed
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1297
    def _get_or_create_wf(self, etypes):
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1298
        if not isinstance(etypes, (list, tuple)):
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1299
            etypes = (etypes,)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1300
        rset = self.rqlexec('Workflow X WHERE X workflow_of ET, ET name %(et)s',
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1301
                            {'et': etypes[0]})
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1302
        if rset:
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1303
            return rset.get_entity(0, 0)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1304
        return self.cmd_add_workflow('%s workflow' % ';'.join(etypes), etypes)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1305
4339
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1306
    @deprecated('[3.5] use add_workflow and Workflow.add_state method',
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1307
                stacklevel=3)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1308
    def cmd_add_state(self, name, stateof, initial=False, commit=False, **kwargs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1309
        """method to ease workflow definition: add a state for one or more
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1310
        entity type(s)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1311
        """
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1312
        wf = self._get_or_create_wf(stateof)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1313
        state = wf.add_state(name, initial, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1314
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1315
            self.commit()
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1316
        return state.eid
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1317
4339
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1318
    @deprecated('[3.5] use add_workflow and Workflow.add_transition method',
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1319
                stacklevel=3)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1320
    def cmd_add_transition(self, name, transitionof, fromstates, tostate,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1321
                           requiredgroups=(), conditions=(), commit=False, **kwargs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1322
        """method to ease workflow definition: add a transition for one or more
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1323
        entity type(s), from one or more state and to a single state
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1324
        """
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1325
        wf = self._get_or_create_wf(transitionof)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1326
        tr = wf.add_transition(name, fromstates, tostate, requiredgroups,
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1327
                               conditions, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1328
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1329
            self.commit()
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1330
        return tr.eid
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1331
4339
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1332
    @deprecated('[3.5] use Transition.set_transition_permissions method',
fe93b670343d some fixes so that deprecation warning are properly localized
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4330
diff changeset
  1333
                stacklevel=3)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1334
    def cmd_set_transition_permissions(self, treid,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1335
                                       requiredgroups=(), conditions=(),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1336
                                       reset=True, commit=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1337
        """set or add (if `reset` is False) groups and conditions for a
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1338
        transition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1339
        """
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1340
        tr = self._cw.entity_from_eid(treid)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1341
        tr.set_transition_permissions(requiredgroups, conditions, reset)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1342
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1343
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1344
5556
9ab2b4c74baf [entity] introduce a new 'adapters' registry
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1345
    @deprecated('[3.5] use iworkflowable.fire_transition("transition") or '
9ab2b4c74baf [entity] introduce a new 'adapters' registry
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1346
                'iworkflowable.change_state("state")', stacklevel=3)
972
1efba3fd1364 set_state command in shell
sylvain.thenault@logilab.fr
parents: 934
diff changeset
  1347
    def cmd_set_state(self, eid, statename, commit=False):
5556
9ab2b4c74baf [entity] introduce a new 'adapters' registry
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1348
        self._cw.entity_from_eid(eid).cw_adapt_to('IWorkflowable').change_state(statename)
972
1efba3fd1364 set_state command in shell
sylvain.thenault@logilab.fr
parents: 934
diff changeset
  1349
        if commit:
1efba3fd1364 set_state command in shell
sylvain.thenault@logilab.fr
parents: 934
diff changeset
  1350
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1351
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1352
    # CWProperty handling ######################################################
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1353
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1354
    def cmd_property_value(self, pkey):
7111
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1355
        """retreive the site-wide persistent property value for the given key.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1356
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1357
        To get a user specific property value, use appropriate method on CWUser
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1358
        instance.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1359
        """
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1360
        rset = self.rqlexec(
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1361
            'Any V WHERE X is CWProperty, X pkey %(k)s, X value V, NOT X for_user U',
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1362
            {'k': pkey}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1363
        return rset[0][0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1364
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1365
    def cmd_set_property(self, pkey, value):
7111
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1366
        """set the site-wide persistent property value for the given key to the
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1367
        given value.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1368
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1369
        To set a user specific property value, use appropriate method on CWUser
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1370
        instance.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1371
        """
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1372
        value = unicode(value)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1373
        try:
7111
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1374
            prop = self.rqlexec(
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1375
                'CWProperty X WHERE X pkey %(k)s, NOT X for_user U',
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1376
                {'k': pkey}, ask_confirm=False).get_entity(0, 0)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
  1377
        except Exception:
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1378
            self.cmd_create_entity('CWProperty', pkey=unicode(pkey), value=value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1379
        else:
7111
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1380
            prop.set_attributes(value=value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1381
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1382
    # other data migration commands ###########################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1383
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1384
    @property
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1385
    def _cw(self):
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1386
        session = self.session
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1387
        if session is not None:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1388
            session.set_cnxset()
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1389
            return session
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1390
        return self.cnx.request()
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
  1391
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1392
    def cmd_storage_changed(self, etype, attribute):
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1393
        """migrate entities to a custom storage. The new storage is expected to
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1394
        be set, it will be temporarily removed for the migration.
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1395
        """
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1396
        from logilab.common.shellutils import ProgressBar
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1397
        source = self.repo.system_source
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1398
        storage = source.storage(etype, attribute)
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1399
        source.unset_storage(etype, attribute)
5552
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5551
diff changeset
  1400
        rset = self.rqlexec('Any X WHERE X is %s' % etype, ask_confirm=False)
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1401
        pb = ProgressBar(len(rset))
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1402
        for entity in rset.entities():
5551
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1403
            # fill cache. Do not fetch that attribute using the global rql query
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1404
            # since we may exhaust memory doing that....
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1405
            getattr(entity, attribute)
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1406
            storage.migrate_entity(entity, attribute)
5551
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1407
            # remove from entity cache to avoid memory exhaustion
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1408
            del entity[attribute]
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1409
            pb.update()
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1410
        print
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1411
        source.set_storage(etype, attribute, storage)
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1412
5132
260d73ad4f24 [cleaning] simpler implementation of cmd_create_entity
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5043
diff changeset
  1413
    def cmd_create_entity(self, etype, commit=False, **kwargs):
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1414
        """add a new entity of the given type"""
3708
95e8c3a9698a update prototype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3707
diff changeset
  1415
        entity = self._cw.create_entity(etype, **kwargs)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1416
        if commit:
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1417
            self.commit()
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1418
        return entity
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1419
7116
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1420
    def cmd_find_entities(self, etype, **kwargs):
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1421
        """find entities of the given type and attribute values"""
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1422
        return self._cw.find_entities(etype, **kwargs)
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1423
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1424
    def cmd_find_one_entity(self, etype, **kwargs):
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1425
        """find one entity of the given type and attribute values.
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1426
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1427
        raise :exc:`cubicweb.req.FindEntityError` if can not return one and only
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1428
        one entity.
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1429
        """
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1430
        return self._cw.find_one_entity(etype, **kwargs)
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1431
5928
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1432
    def cmd_update_etype_fti_weight(self, etype, weight):
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1433
        if self.repo.system_source.dbdriver == 'postgres':
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1434
            self.sqlexec('UPDATE appears SET weight=%(weight)s '
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1435
                         'FROM entities as X '
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1436
                         'WHERE X.eid=appears.uid AND X.type=%(type)s',
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1437
                         {'type': etype, 'weight': weight}, ask_confirm=False)
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1438
5850
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1439
    def cmd_reindex_entities(self, etypes=None):
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1440
        """force reindexaction of entities of the given types or of all
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1441
        indexable entity types
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1442
        """
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1443
        from cubicweb.server.checkintegrity import reindex_entities
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1444
        reindex_entities(self.repo.schema, self.session, etypes=etypes)
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1445
4367
fa02f7dccfe4 fix deprecation warnings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4344
diff changeset
  1446
    @deprecated('[3.5] use create_entity', stacklevel=3)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1447
    def cmd_add_entity(self, etype, *args, **kwargs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1448
        """add a new entity of the given type"""
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1449
        return self.cmd_create_entity(etype, *args, **kwargs).eid
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1450
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1451
    @contextmanager
7409
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1452
    def cmd_dropped_constraints(self, etype, attrname, cstrtype=None,
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1453
                                droprequired=False):
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1454
        """context manager to drop constraints temporarily on fs_schema
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1455
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1456
        `cstrtype` should be a constraint class (or a tuple of classes)
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1457
        and will be passed to isinstance directly
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1458
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1459
        For instance::
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1460
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1461
            >>> with dropped_constraints('MyType', 'myattr',
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1462
            ...                          UniqueConstraint, droprequired=True):
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1463
            ...     add_attribute('MyType', 'myattr')
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1464
            ...     # + instructions to fill MyType.myattr column
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1465
            ...
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1466
            >>>
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1467
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1468
        """
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1469
        rdef = self.fs_schema.eschema(etype).rdef(attrname)
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1470
        original_constraints = rdef.constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1471
        # remove constraints
7409
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1472
        if cstrtype:
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1473
            rdef.constraints = [cstr for cstr in original_constraints
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1474
                                if not (cstrtype and isinstance(cstr, cstrtype))]
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1475
        if droprequired:
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1476
            original_cardinality = rdef.cardinality
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1477
            rdef.cardinality = '?' + rdef.cardinality[1]
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1478
        yield
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1479
        # restore original constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1480
        rdef.constraints = original_constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1481
        if droprequired:
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1482
            rdef.cardinality = original_cardinality
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1483
        # update repository schema
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1484
        self.cmd_sync_schema_props_perms(rdef, syncperms=False)
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1485
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1486
    def sqlexec(self, sql, args=None, ask_confirm=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1487
        """execute the given sql if confirmed
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1488
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1489
        should only be used for low level stuff undoable with existing higher
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1490
        level actions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1491
        """
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1492
        if not ask_confirm or self.confirm('Execute sql: %s ?' % sql):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1493
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1494
                cu = self.session.system_sql(sql, args)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
  1495
            except Exception:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1496
                ex = sys.exc_info()[1]
6187
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6186
diff changeset
  1497
                if self.confirm('Error: %s\nabort?' % ex, pdb=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1498
                    raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1499
                return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1500
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1501
                return cu.fetchall()
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
  1502
            except Exception:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1503
                # no result to fetch
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1504
                return
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1505
4945
356662a6f06c [migration] new build_descr argument to rqlexec on the migration helper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4925
diff changeset
  1506
    def rqlexec(self, rql, kwargs=None, cachekey=None, build_descr=True,
6186
8ada7df95877 [migration] set ask_confirm=False by default on rqlexec
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6179
diff changeset
  1507
                ask_confirm=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1508
        """rql action"""
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1509
        if cachekey is not None:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1510
            warn('[3.8] cachekey is deprecated, you can safely remove this argument',
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1511
                 DeprecationWarning, stacklevel=2)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1512
        if not isinstance(rql, (tuple, list)):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1513
            rql = ( (rql, kwargs), )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1514
        res = None
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1515
        execute = self._cw.execute
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1516
        for rql, kwargs in rql:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1517
            if kwargs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1518
                msg = '%s (%s)' % (rql, kwargs)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1519
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1520
                msg = rql
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1521
            if not ask_confirm or self.confirm('Execute rql: %s ?' % msg):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1522
                try:
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1523
                    res = execute(rql, kwargs, build_descr=build_descr)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1524
                except Exception, ex:
6187
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6186
diff changeset
  1525
                    if self.confirm('Error: %s\nabort?' % ex, pdb=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1526
                        raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1527
        return res
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1528
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1529
    def rqliter(self, rql, kwargs=None, ask_confirm=True):
6686
070e3b23160d [migration] fix rqliter when some query arguments are given
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6582
diff changeset
  1530
        return ForRqlIterator(self, rql, kwargs, ask_confirm)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1531
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1532
    # broken db commands ######################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1533
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1534
    def cmd_change_attribute_type(self, etype, attr, newtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1535
        """low level method to change the type of an entity attribute. This is
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1536
        a quick hack which has some drawback:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1537
        * only works when the old type can be changed to the new type by the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1538
          underlying rdbms (eg using ALTER TABLE)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1539
        * the actual schema won't be updated until next startup
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1540
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1541
        rschema = self.repo.schema.rschema(attr)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1542
        oldtype = rschema.objects(etype)[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1543
        rdefeid = rschema.rproperty(etype, oldtype, 'eid')
7410
1dbf9a83bb49 [migractions] fix change_attribute_type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7409
diff changeset
  1544
        sql = ("UPDATE cw_CWAttribute "
1dbf9a83bb49 [migractions] fix change_attribute_type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7409
diff changeset
  1545
               "SET cw_to_entity=(SELECT cw_eid FROM cw_CWEType WHERE cw_name='%s')"
1dbf9a83bb49 [migractions] fix change_attribute_type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7409
diff changeset
  1546
               "WHERE cw_eid=%s") % (newtype, rdefeid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1547
        self.sqlexec(sql, ask_confirm=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1548
        dbhelper = self.repo.system_source.dbhelper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1549
        sqltype = dbhelper.TYPE_MAPPING[newtype]
7410
1dbf9a83bb49 [migractions] fix change_attribute_type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7409
diff changeset
  1550
        sql = 'ALTER TABLE cw_%s ALTER COLUMN cw_%s TYPE %s' % (etype, attr, sqltype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1551
        self.sqlexec(sql, ask_confirm=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1552
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1553
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1554
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1555
    def cmd_add_entity_type_table(self, etype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1556
        """low level method to create the sql table for an existing entity.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1557
        This may be useful on accidental desync between the repository schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1558
        and a sql database
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1559
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1560
        dbhelper = self.repo.system_source.dbhelper
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
  1561
        tablesql = eschema2sql(dbhelper, self.repo.schema.eschema(etype),
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
  1562
                               prefix=SQL_PREFIX)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1563
        for sql in tablesql.split(';'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1564
            if sql.strip():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1565
                self.sqlexec(sql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1566
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1567
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1568
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1569
    def cmd_add_relation_type_table(self, rtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1570
        """low level method to create the sql table for an existing relation.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1571
        This may be useful on accidental desync between the repository schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1572
        and a sql database
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1573
        """
7879
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
  1574
        tablesql = rschema2sql(self.repo.schema.rschema(rtype))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1575
        for sql in tablesql.split(';'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1576
            if sql.strip():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1577
                self.sqlexec(sql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1578
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1579
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1580
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
  1581
    @deprecated("[3.7] use session.disable_hook_categories('integrity')")
4834
b718626a0e60 move hooks activation control on session object, so we can have a per transaction control. Added a new `hooks_control` context manager for usual modification of hooks activation.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4763
diff changeset
  1582
    def cmd_deactivate_verification_hooks(self):
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
  1583
        self.session.disable_hook_categories('integrity')
4834
b718626a0e60 move hooks activation control on session object, so we can have a per transaction control. Added a new `hooks_control` context manager for usual modification of hooks activation.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4763
diff changeset
  1584
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
  1585
    @deprecated("[3.7] use session.enable_hook_categories('integrity')")
4834
b718626a0e60 move hooks activation control on session object, so we can have a per transaction control. Added a new `hooks_control` context manager for usual modification of hooks activation.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4763
diff changeset
  1586
    def cmd_reactivate_verification_hooks(self):
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
  1587
        self.session.enable_hook_categories('integrity')
4834
b718626a0e60 move hooks activation control on session object, so we can have a per transaction control. Added a new `hooks_control` context manager for usual modification of hooks activation.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4763
diff changeset
  1588
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1589
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1590
class ForRqlIterator:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1591
    """specific rql iterator to make the loop skipable"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1592
    def __init__(self, helper, rql, kwargs, ask_confirm):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1593
        self._h = helper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1594
        self.rql = rql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1595
        self.kwargs = kwargs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1596
        self.ask_confirm = ask_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1597
        self._rsetit = None
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1598
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1599
    def __iter__(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1600
        return self
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1601
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1602
    def _get_rset(self):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1603
        rql, kwargs = self.rql, self.kwargs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1604
        if kwargs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1605
            msg = '%s (%s)' % (rql, kwargs)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1606
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1607
            msg = rql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1608
        if self.ask_confirm:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1609
            if not self._h.confirm('Execute rql: %s ?' % msg):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1610
                raise StopIteration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1611
        try:
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1612
            return self._h._cw.execute(rql, kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1613
        except Exception, ex:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1614
            if self._h.confirm('Error: %s\nabort?' % ex):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1615
                raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1616
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1617
                raise StopIteration
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1618
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1619
    def next(self):
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1620
        if self._rsetit is not None:
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1621
            return self._rsetit.next()
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1622
        rset = self._get_rset()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1623
        self._rsetit = iter(rset)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1624
        return self._rsetit.next()
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1625
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1626
    def entities(self):
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1627
        try:
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1628
            rset = self._get_rset()
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1629
        except StopIteration:
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1630
            return []
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1631
        return rset.entities()