server/migractions.py
author Rémi Cardona <remi.cardona@logilab.fr>
Tue, 13 Oct 2015 15:09:22 +0200
changeset 10663 54b8a1f249fb
parent 10662 10942ed172de
child 10669 155c29e0ed1c
permissions -rw-r--r--
[py3k] dict.itervalues → dict.values
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
9588
fe267b7336f3 [migration] always rebuild infered relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9421
diff changeset
     1
# copyright 2003-2014 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5399
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
"""a class implementing basic actions used in migration scripts.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
The following schema actions are supported for now:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
* add/drop/rename attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
* add/drop entity/relation type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
* rename entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
The following data actions are supported for now:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
* add an entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
* execute raw RQL queries
5850
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
    28
"""
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
    29
from __future__ import print_function
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
    30
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    31
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    32
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    33
import sys
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    34
import os
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    35
import tarfile
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    36
import tempfile
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    37
import shutil
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
    38
import os.path as osp
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 972
diff changeset
    39
from datetime import datetime
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
    40
from glob import glob
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
    41
from copy import copy
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
    42
from warnings import warn
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
    43
from contextlib import contextmanager
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    44
2613
5e19c2bb370e R [all] logilab.common 0.44 provides only deprecated
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2493
diff changeset
    45
from logilab.common.deprecation import deprecated
2107
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    46
from logilab.common.decorators import cached, clear_cache
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    47
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    48
from yams.constraints import SizeConstraint
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
    49
from yams.schema import RelationDefinitionSchema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
    51
from cubicweb import CW_SOFTWARE_ROOT, AuthenticationError, ExecutionError
8190
2a3c1b787688 [vreg] move base registry implementation to logilab.common. Closes #1916014
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8181
diff changeset
    52
from cubicweb.predicates import is_instance
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
    53
from cubicweb.schema import (ETYPE_NAME_MAP, META_RTYPES, VIRTUAL_RTYPES,
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
    54
                             PURE_VIRTUAL_RTYPES,
2926
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
    55
                             CubicWebRelationSchema, order_eschemas)
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    56
from cubicweb.cwvreg import CW_EVENT_MANAGER
9511
241b1232ed7f Use repoapi instead of dbapi for cwctl shell, upgrade and db-init
Julien Cristau <julien.cristau@logilab.fr>
parents: 9478
diff changeset
    57
from cubicweb import repoapi
4021
280c910c8710 move i18n / migration modules from cw.common to cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4017
diff changeset
    58
from cubicweb.migration import MigrationHelper, yes
9468
39b7a91a3f4c [repo] pylint cleanup, mainly of imports, with a bit of style
Julien Cristau <julien.cristau@logilab.fr>
parents: 9463
diff changeset
    59
from cubicweb.server import hook, schemaserial as ss
10200
cceb2c7c02f4 Use our version of schema2sql
Julien Cristau <julien.cristau@logilab.fr>
parents: 10129
diff changeset
    60
from cubicweb.server.schema2sql import eschema2sql, rschema2sql, unique_index_name
9468
39b7a91a3f4c [repo] pylint cleanup, mainly of imports, with a bit of style
Julien Cristau <julien.cristau@logilab.fr>
parents: 9463
diff changeset
    61
from cubicweb.server.utils import manager_userpasswd
39b7a91a3f4c [repo] pylint cleanup, mainly of imports, with a bit of style
Julien Cristau <julien.cristau@logilab.fr>
parents: 9463
diff changeset
    62
from cubicweb.server.sqlutils import sqlexec, SQL_PREFIX
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
6571
71b446d1cf93 [dependancy] do not use testlib from outside test: recent lgc versions recommends unittest2 while lgc.testlib needs it, and we don't want it to be necessary to run cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6323
diff changeset
    64
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    65
class ClearGroupMap(hook.Hook):
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    66
    __regid__ = 'cw.migration.clear_group_mapping'
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    67
    __select__ = hook.Hook.__select__ & is_instance('CWGroup')
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    68
    events = ('after_add_entity', 'after_update_entity',)
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    69
    def __call__(self):
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    70
        clear_cache(self.mih, 'group_mapping')
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
    71
        self.mih._synchronized.clear()
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
    72
6205
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    73
    @classmethod
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    74
    def mih_register(cls, repo):
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    75
        # may be already registered in tests (e.g. unittest_migractions at
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    76
        # least)
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    77
        if not cls.__regid__ in repo.vreg['after_add_entity_hooks']:
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    78
            repo.vreg.register(ClearGroupMap)
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
    79
9630
e7dbc4f06a48 minor cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9606
diff changeset
    80
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    81
class ServerMigrationHelper(MigrationHelper):
9175
a7412e884d7b fix typos in docstring, doc and comments
Julien Cristau <julien.cristau@logilab.fr>
parents: 9164
diff changeset
    82
    """specific migration helper for server side migration scripts,
a7412e884d7b fix typos in docstring, doc and comments
Julien Cristau <julien.cristau@logilab.fr>
parents: 9164
diff changeset
    83
    providing actions related to schema/data migration
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    84
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    85
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
    def __init__(self, config, schema, interactive=True,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
                 repo=None, cnx=None, verbosity=1, connect=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    88
        MigrationHelper.__init__(self, config, interactive, verbosity)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    89
        if not interactive:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
            assert cnx
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
            assert repo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
        if cnx is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
            assert repo
9556
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
    94
            self.cnx = cnx
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
            self.repo = repo
10355
60b8204fcca3 [session] all cnx._session become cnx.session
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10353
diff changeset
    96
            self.session = cnx.session
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    97
        elif connect:
10569
af47954c1015 [migration] remove repo_connect and use config.repository() instead
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10553
diff changeset
    98
            self.repo = config.repository()
10346
b926ff4ef4a8 [repoapi,session] remove all session-as-cnx backward compat
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10301
diff changeset
    99
            self.set_cnx()
9556
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   100
        else:
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   101
            self.session = None
5019
72734c210836 [c-c] new server_maintenance hook, called on c-c shell / upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   102
        # no config on shell to a remote instance
5034
4781870e97d9 [maintainance] don't crash if we've no in-memory repository
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5019
diff changeset
   103
        if config is not None and (cnx or connect):
6203
d3dea5f84404 [test] during some tests (unittest_migrations), we should not register ClearGroupMap which is actually already there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6200
diff changeset
   104
            repo = self.repo
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   105
            # register a hook to clear our group_mapping cache and the
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   106
            # self._synchronized set when some group is added or updated
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   107
            ClearGroupMap.mih = self
6205
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
   108
            ClearGroupMap.mih_register(repo)
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
   109
            CW_EVENT_MANAGER.bind('after-registry-reload',
41e0e13e10b7 [test] fix in 6203:d3dea5f84404 wasn't enough
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6203
diff changeset
   110
                                  ClearGroupMap.mih_register, repo)
6200
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   111
            # notify we're starting maintenance (called instead of server_start
6e8c847ae397 [migration] when some CWGroup is added during migration, we should reset migration handler's group mapping cache and the set telling what have been synchronized (we may be able to add some new groups on perms syncing...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6190
diff changeset
   112
            # which is called on regular start
6203
d3dea5f84404 [test] during some tests (unittest_migrations), we should not register ClearGroupMap which is actually already there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6200
diff changeset
   113
            repo.hm.call_hooks('server_maintenance', repo=repo)
10084
eb3681e13ed9 [config] kill a getattr
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10014
diff changeset
   114
        if not schema and not config.quick_start:
7797
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   115
            insert_lperms = self.repo.get_versions()['cubicweb'] < (3, 14, 0) and 'localperms' in config.available_cubes()
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   116
            if insert_lperms:
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   117
                cubes = config._cubes
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   118
                config._cubes += ('localperms',)
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   119
            try:
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   120
                schema = config.load_schema(expand_cubes=True)
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   121
            finally:
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   122
                if insert_lperms:
a71618a75b53 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   123
                    config._cubes = cubes
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   124
        self.fs_schema = schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   125
        self._synchronized = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   126
10353
d9a1e7939ee6 [migractions] remove any session related leftovers
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10346
diff changeset
   127
    # overriden from base MigrationHelper ######################################
d9a1e7939ee6 [migractions] remove any session related leftovers
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10346
diff changeset
   128
10346
b926ff4ef4a8 [repoapi,session] remove all session-as-cnx backward compat
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10301
diff changeset
   129
    def set_cnx(self):
9556
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   130
        try:
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   131
            login = self.repo.config.default_admin_config['login']
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   132
            pwd = self.repo.config.default_admin_config['password']
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   133
        except KeyError:
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   134
            login, pwd = manager_userpasswd()
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   135
        while True:
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   136
            try:
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   137
                self.cnx = repoapi.connect(self.repo, login, password=pwd)
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   138
                if not 'managers' in self.cnx.user.groups:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   139
                    print('migration need an account in the managers group')
9556
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   140
                else:
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   141
                    break
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   142
            except AuthenticationError:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   143
                print('wrong user/password')
9556
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   144
            except (KeyboardInterrupt, EOFError):
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   145
                print('aborting...')
9556
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   146
                sys.exit(0)
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   147
            try:
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   148
                login, pwd = manager_userpasswd()
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   149
            except (KeyboardInterrupt, EOFError):
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   150
                print('aborting...')
9556
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   151
                sys.exit(0)
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   152
        self.session = self.repo._get_session(self.cnx.sessionid)
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   153
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   154
    def cube_upgraded(self, cube, version):
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   155
        self.cmd_set_property('system.version.%s' % cube.lower(),
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   156
                              unicode(version))
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   157
        self.commit()
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   158
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   159
    def shutdown(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
        if self.repo is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   161
            self.repo.shutdown()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   162
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   163
    def migrate(self, vcconf, toupgrade, options):
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   164
        if not options.fs_only:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   165
            if options.backup_db is None:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   166
                self.backup_database()
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   167
            elif options.backup_db:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   168
                self.backup_database(askconfirm=False)
4925
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
   169
        # disable notification during migration
9556
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   170
        with self.cnx.allow_all_hooks_but('notification'):
4925
0d66fbe050c6 [migration] disable notification by default during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4905
diff changeset
   171
            super(ServerMigrationHelper, self).migrate(vcconf, toupgrade, options)
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   172
3715
e3ccadb126d7 [shell] make process_script available throuhg c-c shell / migration script context
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3708
diff changeset
   173
    def cmd_process_script(self, migrscript, funcname=None, *args, **kwargs):
10365
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10355
diff changeset
   174
        try:
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10355
diff changeset
   175
            return super(ServerMigrationHelper, self).cmd_process_script(
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10355
diff changeset
   176
                  migrscript, funcname, *args, **kwargs)
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10355
diff changeset
   177
        except ExecutionError as err:
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10355
diff changeset
   178
            sys.stderr.write("-> %s\n" % err)
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10355
diff changeset
   179
        except BaseException:
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10355
diff changeset
   180
            self.rollback()
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10355
diff changeset
   181
            raise
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   182
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   183
    # Adjust docstring
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   184
    cmd_process_script.__doc__ = MigrationHelper.cmd_process_script.__doc__
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5999
diff changeset
   185
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   186
    # server specific migration methods ########################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   187
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   188
    def backup_database(self, backupfile=None, askconfirm=True, format='native'):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   189
        config = self.config
10569
af47954c1015 [migration] remove repo_connect and use config.repository() instead
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10553
diff changeset
   190
        repo = self.repo
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   191
        # paths
3192
93c8fdcd943e windows doesn't like ':' in filesc
Aurelien Campeas
parents: 3120
diff changeset
   192
        timestamp = datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   193
        instbkdir = osp.join(config.appdatahome, 'backup')
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   194
        if not osp.exists(instbkdir):
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   195
            os.makedirs(instbkdir)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   196
        backupfile = backupfile or osp.join(instbkdir, '%s-%s.tar.gz'
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   197
                                            % (config.appid, timestamp))
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   198
        # check backup has to be done
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   199
        if osp.exists(backupfile) and not \
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   200
                self.confirm('Backup file %s exists, overwrite it?' % backupfile):
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   201
            print('-> no backup done.')
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   202
            return
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   203
        elif askconfirm and not self.confirm('Backup %s database?' % config.appid):
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   204
            print('-> no backup done.')
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   205
            return
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   206
        open(backupfile,'w').close() # kinda lock
10591
8e46ed1a0b8a [py3k] octals and long
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10589
diff changeset
   207
        os.chmod(backupfile, 0o600)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   208
        # backup
9456
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   209
        source = repo.system_source
6918
9e607157d4cf [cw-ctl] use default `mkdtemp()` behaviour to store backup file internally
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6816
diff changeset
   210
        tmpdir = tempfile.mkdtemp()
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   211
        try:
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   212
            failed = False
9456
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   213
            try:
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   214
                source.backup(osp.join(tmpdir, source.uri), self.confirm, format=format)
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   215
            except Exception as ex:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   216
                print('-> error trying to backup %s [%s]' % (source.uri, ex))
9456
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   217
                if not self.confirm('Continue anyway?', default='n'):
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   218
                    raise SystemExit(1)
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   219
                else:
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   220
                    failed = True
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   221
            with open(osp.join(tmpdir, 'format.txt'), 'w') as format_file:
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   222
                format_file.write('%s\n' % format)
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   223
            with open(osp.join(tmpdir, 'versions.txt'), 'w') as version_file:
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   224
                versions = repo.get_versions()
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10651
diff changeset
   225
                for cube, version in versions.items():
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   226
                    version_file.write('%s %s\n' % (cube, version))
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   227
            if not failed:
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   228
                bkup = tarfile.open(backupfile, 'w|gz')
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   229
                for filename in os.listdir(tmpdir):
4721
8f63691ccb7f pylint style fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   230
                    bkup.add(osp.join(tmpdir, filename), filename)
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   231
                bkup.close()
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   232
                # call hooks
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   233
                repo.hm.call_hooks('server_backup', repo=repo, timestamp=timestamp)
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   234
                # done
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   235
                print('-> backup file',  backupfile)
2960
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   236
        finally:
1c6eafc68586 [db-dump] don't create tarball on failed dump, properly remove temporary directory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   237
            shutil.rmtree(tmpdir)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   238
9457
d5ed6efd6448 [c-c restore] drop useless restore-all option, and related systemonly argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9456
diff changeset
   239
    def restore_database(self, backupfile, drop=True, askconfirm=True, format='native'):
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   240
        # check
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   241
        if not osp.exists(backupfile):
6323
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   242
            raise ExecutionError("Backup file %s doesn't exist" % backupfile)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   243
        if askconfirm and not self.confirm('Restore %s database from %s ?'
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2926
diff changeset
   244
                                           % (self.config.appid, backupfile)):
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   245
            return
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   246
        # unpack backup
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   247
        tmpdir = tempfile.mkdtemp()
3105
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   248
        try:
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   249
            bkup = tarfile.open(backupfile, 'r|gz')
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   250
        except tarfile.ReadError:
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   251
            # assume restoring old backup
4344
066e7884e57d add source in backup/restore failure message
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4339
diff changeset
   252
            shutil.copy(backupfile, osp.join(tmpdir, 'system'))
3105
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   253
        else:
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   254
            for name in bkup.getnames():
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   255
                if name[0] in '/.':
6323
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   256
                    raise ExecutionError('Security check failed, path starts with "/" or "."')
3105
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   257
            bkup.close() # XXX seek error if not close+open !?!
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   258
            bkup = tarfile.open(backupfile, 'r|gz')
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   259
            bkup.extractall(path=tmpdir)
b788903e77d5 be able to restore pre cw 3.4 database dumps #370595
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 2963
diff changeset
   260
            bkup.close()
7342
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   261
        if osp.isfile(osp.join(tmpdir, 'format.txt')):
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   262
            with open(osp.join(tmpdir, 'format.txt')) as format_file:
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   263
                written_format = format_file.readline().strip()
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   264
                if written_format in ('portable', 'native'):
d1c8b5b3531c adds support for a portable db import/export format (closes: #1521112)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7295
diff changeset
   265
                    format = written_format
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   266
        self.config.init_cnxset_pool = False
10569
af47954c1015 [migration] remove repo_connect and use config.repository() instead
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10553
diff changeset
   267
        repo = self.repo
9456
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   268
        source = repo.system_source
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   269
        try:
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   270
            source.restore(osp.join(tmpdir, source.uri), self.confirm, drop, format)
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   271
        except Exception as exc:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   272
            print('-> error trying to restore %s [%s]' % (source.uri, exc))
9456
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   273
            if not self.confirm('Continue anyway?', default='n'):
a79e88aad555 [multi-sources-removal] Kill repo.sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9451
diff changeset
   274
                raise SystemExit(1)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   275
        shutil.rmtree(tmpdir)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   276
        # call hooks
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
   277
        repo.init_cnxset_pool()
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   278
        repo.hm.call_hooks('server_restore', repo=repo, timestamp=backupfile)
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   279
        print('-> database restored.')
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   280
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   281
    def commit(self):
10353
d9a1e7939ee6 [migractions] remove any session related leftovers
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10346
diff changeset
   282
        self.cnx.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   283
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   284
    def rollback(self):
10353
d9a1e7939ee6 [migractions] remove any session related leftovers
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10346
diff changeset
   285
        self.cnx.rollback()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   286
6186
8ada7df95877 [migration] set ask_confirm=False by default on rqlexec
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6179
diff changeset
   287
    def rqlexecall(self, rqliter, ask_confirm=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   288
        for rql, kwargs in rqliter:
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   289
            self.rqlexec(rql, kwargs, ask_confirm=ask_confirm)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   291
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   292
    def _create_context(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   293
        """return a dictionary to use as migration script execution context"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   294
        context = super(ServerMigrationHelper, self)._create_context()
4017
a5b4d4f2a1c7 use commit in migration script instead of checkpoint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4011
diff changeset
   295
        context.update({'commit': self.checkpoint,
4330
cb7b68679501 make rollback available in shell (as commit is)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   296
                        'rollback': self.rollback,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   297
                        'sql': self.sqlexec,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   298
                        'rql': self.rqlexec,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   299
                        'rqliter': self.rqliter,
3707
78596919ede3 [c-c] fixes for shell w/ pyro instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3700
diff changeset
   300
                        'schema': self.repo.get_schema(),
78596919ede3 [c-c] fixes for shell w/ pyro instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3700
diff changeset
   301
                        'cnx': self.cnx,
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   302
                        'fsschema': self.fs_schema,
10353
d9a1e7939ee6 [migractions] remove any session related leftovers
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10346
diff changeset
   303
                        'session' : self.cnx,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   304
                        'repo' : self.repo,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   305
                        })
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   306
        return context
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   307
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   308
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   309
    def group_mapping(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   310
        """cached group mapping"""
9511
241b1232ed7f Use repoapi instead of dbapi for cwctl shell, upgrade and db-init
Julien Cristau <julien.cristau@logilab.fr>
parents: 9478
diff changeset
   311
        return ss.group_mapping(self.cnx)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   312
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   313
    def cstrtype_mapping(self):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   314
        """cached constraint types mapping"""
9511
241b1232ed7f Use repoapi instead of dbapi for cwctl shell, upgrade and db-init
Julien Cristau <julien.cristau@logilab.fr>
parents: 9478
diff changeset
   315
        return ss.cstrtype_mapping(self.cnx)
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   316
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   317
    def cmd_exec_event_script(self, event, cube=None, funcname=None,
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   318
                              *args, **kwargs):
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   319
        """execute a cube event scripts  `migration/<event>.py` where event
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   320
        is one of 'precreate', 'postcreate', 'preremove' and 'postremove'.
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   321
        """
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   322
        assert event in ('precreate', 'postcreate', 'preremove', 'postremove')
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   323
        if cube:
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   324
            cubepath = self.config.cube_dir(cube)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   325
            apc = osp.join(cubepath, 'migration', '%s.py' % event)
7974
77eec6d6e144 [test] fix regression introduced in 7915:a7f3245e1728 leading to test's postcreate not being executed anymore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7948
diff changeset
   326
        elif kwargs.pop('apphome', False):
77eec6d6e144 [test] fix regression introduced in 7915:a7f3245e1728 leading to test's postcreate not being executed anymore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7948
diff changeset
   327
            apc = osp.join(self.config.apphome, 'migration', '%s.py' % event)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
        else:
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   329
            apc = osp.join(self.config.migration_scripts_dir(), '%s.py' % event)
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2700
diff changeset
   330
        if osp.exists(apc):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
            if self.config.free_wheel:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2788
diff changeset
   332
                self.cmd_deactivate_verification_hooks()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   333
            self.info('executing %s', apc)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   334
            confirm = self.confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   335
            execscript_confirm = self.execscript_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   336
            self.confirm = yes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   337
            self.execscript_confirm = yes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   338
            try:
8210
1d1cfc97f6b9 [migration] fire hooks in postcreate by default (closes #1417110)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8190
diff changeset
   339
                if event == 'postcreate':
9556
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   340
                    with self.cnx.allow_all_hooks_but():
8210
1d1cfc97f6b9 [migration] fire hooks in postcreate by default (closes #1417110)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8190
diff changeset
   341
                        return self.cmd_process_script(apc, funcname, *args, **kwargs)
3715
e3ccadb126d7 [shell] make process_script available throuhg c-c shell / migration script context
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3708
diff changeset
   342
                return self.cmd_process_script(apc, funcname, *args, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   343
            finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   344
                self.confirm = confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
                self.execscript_confirm = execscript_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   346
                if self.config.free_wheel:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2788
diff changeset
   347
                    self.cmd_reactivate_verification_hooks()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   348
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   349
    def cmd_install_custom_sql_scripts(self, cube=None):
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   350
        """install a cube custom sql scripts `schema/*.<driver>.sql` where
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   351
        <driver> depends on the instance main database backend (eg 'postgres',
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   352
        'mysql'...)
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   353
        """
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   354
        driver = self.repo.system_source.dbdriver
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   355
        if cube is None:
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   356
            directory = osp.join(CW_SOFTWARE_ROOT, 'schemas')
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   357
        else:
8047
972360b7677b [instance creation] properly search for sql extension in the schema directory. Closes #2068117
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7974
diff changeset
   358
            directory = osp.join(self.config.cube_dir(cube), 'schema')
7990
a673d1d9a738 [diet] drop pre 3.6 API compatibility (but attempt to keep data cmopatibility). Closes #2017916
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7979
diff changeset
   359
        sql_scripts = glob(osp.join(directory, '*.%s.sql' % driver))
7479
b0603fc4ed6d [sql] implement #1631319: ask user before creating postgresql languages
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7412
diff changeset
   360
        for fpath in sql_scripts:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   361
            print('-> installing', fpath)
9556
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   362
            failed = sqlexec(open(fpath).read(), self.cnx.system_sql, False,
9421
6b673cb4d3b6 [migractions] Don't silently ignore errors when installing sql procedures
Julien Cristau <julien.cristau@logilab.fr>
parents: 9375
diff changeset
   363
                             delimiter=';;')
6b673cb4d3b6 [migractions] Don't silently ignore errors when installing sql procedures
Julien Cristau <julien.cristau@logilab.fr>
parents: 9375
diff changeset
   364
            if failed:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   365
                print('-> ERROR, skipping', fpath)
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   366
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   367
    # schema synchronization internals ########################################
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   368
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   369
    def _synchronize_permissions(self, erschema, teid):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   370
        """permission synchronization for an entity or relation type"""
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   371
        assert teid, erschema
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   372
        if 'update' in erschema.ACTIONS or erschema.final:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   373
            # entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   374
            exprtype = u'ERQLExpression'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   375
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   376
            # relation type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   377
            exprtype = u'RRQLExpression'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   378
        gm = self.group_mapping()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   379
        confirm = self.verbosity >= 2
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   380
        # * remove possibly deprecated permission (eg in the persistent schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   381
        #   but not in the new schema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   382
        # * synchronize existing expressions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   383
        # * add new groups/expressions
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   384
        for action in erschema.ACTIONS:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
            perm = '%s_permission' % action
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   386
            # handle groups
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   387
            newgroups = list(erschema.get_groups(action))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   388
            for geid, gname in self.rqlexec('Any G, GN WHERE T %s G, G name GN, '
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   389
                                            'T eid %%(x)s' % perm, {'x': teid},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   390
                                            ask_confirm=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   391
                if not gname in newgroups:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   392
                    if not confirm or self.confirm('Remove %s permission of %s to %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   393
                                                   % (action, erschema, gname)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   394
                        self.rqlexec('DELETE T %s G WHERE G eid %%(x)s, T eid %s'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   395
                                     % (perm, teid),
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   396
                                     {'x': geid}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   397
                else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   398
                    newgroups.remove(gname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   399
            for gname in newgroups:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   400
                if not confirm or self.confirm('Grant %s permission of %s to %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   401
                                               % (action, erschema, gname)):
6075
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   402
                    try:
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   403
                        self.rqlexec('SET T %s G WHERE G eid %%(x)s, T eid %s'
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   404
                                     % (perm, teid),
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   405
                                     {'x': gm[gname]}, ask_confirm=False)
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   406
                    except KeyError:
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   407
                        self.error('can grant %s perm to unexistant group %s',
6ebecb217efe [migration] dont crash if a group doesn't exists (it may not exists yet)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6074
diff changeset
   408
                                   action, gname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   409
            # handle rql expressions
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   410
            newexprs = dict((expr.expression, expr) for expr in erschema.get_rqlexprs(action))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   411
            for expreid, expression in self.rqlexec('Any E, EX WHERE T %s E, E expression EX, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   412
                                                    'T eid %s' % (perm, teid),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   413
                                                    ask_confirm=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   414
                if not expression in newexprs:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   415
                    if not confirm or self.confirm('Remove %s expression for %s permission of %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   416
                                                   % (expression, action, erschema)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
                        # deleting the relation will delete the expression entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   418
                        self.rqlexec('DELETE T %s E WHERE E eid %%(x)s, T eid %s'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   419
                                     % (perm, teid),
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   420
                                     {'x': expreid}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
                else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   422
                    newexprs.pop(expression)
10663
54b8a1f249fb [py3k] dict.itervalues → dict.values
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10662
diff changeset
   423
            for expression in newexprs.values():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   424
                expr = expression.expression
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   425
                if not confirm or self.confirm('Add %s expression for %s permission of %s?'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   426
                                               % (expr, action, erschema)):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   427
                    self.rqlexec('INSERT RQLExpression X: X exprtype %%(exprtype)s, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   428
                                 'X expression %%(expr)s, X mainvars %%(vars)s, T %s X '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   429
                                 'WHERE T eid %%(x)s' % perm,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   430
                                 {'expr': expr, 'exprtype': exprtype,
7161
e3f69df8dac7 fix failing tests before 3.12 release
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7116
diff changeset
   431
                                  'vars': u','.join(sorted(expression.mainvars)),
e3f69df8dac7 fix failing tests before 3.12 release
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7116
diff changeset
   432
                                  'x': teid},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   433
                                 ask_confirm=False)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   434
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   435
    def _synchronize_rschema(self, rtype, syncrdefs=True,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   436
                             syncperms=True, syncprops=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   437
        """synchronize properties of the persistent relation schema against its
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   438
        current definition:
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   439
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   440
        * description
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4367
diff changeset
   441
        * symmetric, meta
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   442
        * inlined
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   443
        * relation definitions if `syncrdefs`
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   444
        * permissions if `syncperms`
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   445
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   446
        physical schema changes should be handled by repository's schema hooks
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   447
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   448
        rtype = str(rtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   449
        if rtype in self._synchronized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   450
            return
10584
743ed2b13a6f [syncschema] only add to the `synchronized` set if all possible updates have been done
Aurelien Campeas <aurelien.campeas@pythonian.fr>
parents: 9773
diff changeset
   451
        if syncrdefs and syncperms and syncprops:
743ed2b13a6f [syncschema] only add to the `synchronized` set if all possible updates have been done
Aurelien Campeas <aurelien.campeas@pythonian.fr>
parents: 9773
diff changeset
   452
            self._synchronized.add(rtype)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   453
        rschema = self.fs_schema.rschema(rtype)
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   454
        reporschema = self.repo.schema.rschema(rtype)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   455
        if syncprops:
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   456
            assert reporschema.eid, reporschema
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   457
            self.rqlexecall(ss.updaterschema2rql(rschema, reporschema.eid),
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   458
                            ask_confirm=self.verbosity>=2)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   459
        if syncrdefs:
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   460
            for subj, obj in rschema.rdefs:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   461
                if (subj, obj) not in reporschema.rdefs:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   462
                    continue
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   463
                if rschema in VIRTUAL_RTYPES:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   464
                    continue
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   465
                self._synchronize_rdef_schema(subj, rschema, obj,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   466
                                              syncprops=syncprops,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   467
                                              syncperms=syncperms)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   468
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   469
    def _synchronize_eschema(self, etype, syncrdefs=True,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   470
                             syncperms=True, syncprops=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   471
        """synchronize properties of the persistent entity schema against
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   472
        its current definition:
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   473
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   474
        * description
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   475
        * internationalizable, fulltextindexed, indexed, meta
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   476
        * relations from/to this entity
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   477
        * __unique_together__
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   478
        * permissions if `syncperms`
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   479
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   480
        etype = str(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   481
        if etype in self._synchronized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   482
            return
10584
743ed2b13a6f [syncschema] only add to the `synchronized` set if all possible updates have been done
Aurelien Campeas <aurelien.campeas@pythonian.fr>
parents: 9773
diff changeset
   483
        if syncrdefs and syncperms and syncprops:
743ed2b13a6f [syncschema] only add to the `synchronized` set if all possible updates have been done
Aurelien Campeas <aurelien.campeas@pythonian.fr>
parents: 9773
diff changeset
   484
            self._synchronized.add(etype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   485
        repoeschema = self.repo.schema.eschema(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   486
        try:
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   487
            eschema = self.fs_schema.eschema(etype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   488
        except KeyError:
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   489
            return # XXX somewhat unexpected, no?...
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   490
        if syncprops:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   491
            repospschema = repoeschema.specializes()
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   492
            espschema = eschema.specializes()
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   493
            if repospschema and not espschema:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   494
                self.rqlexec('DELETE X specializes Y WHERE X is CWEType, X name %(x)s',
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   495
                             {'x': str(repoeschema)}, ask_confirm=False)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   496
            elif not repospschema and espschema:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   497
                self.rqlexec('SET X specializes Y WHERE X is CWEType, X name %(x)s, '
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   498
                             'Y is CWEType, Y name %(y)s',
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   499
                             {'x': str(repoeschema), 'y': str(espschema)},
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   500
                             ask_confirm=False)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   501
            self.rqlexecall(ss.updateeschema2rql(eschema, repoeschema.eid),
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   502
                            ask_confirm=self.verbosity >= 2)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   503
        if syncperms:
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   504
            self._synchronize_permissions(eschema, repoeschema.eid)
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   505
        if syncrdefs:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   506
            for rschema, targettypes, role in eschema.relation_definitions(True):
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   507
                if rschema in VIRTUAL_RTYPES:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   508
                    continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   509
                if role == 'subject':
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   510
                    if not rschema in repoeschema.subject_relations():
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   511
                        continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   512
                    subjtypes, objtypes = [etype], targettypes
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   513
                else: # role == 'object'
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   514
                    if not rschema in repoeschema.object_relations():
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   515
                        continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   516
                    subjtypes, objtypes = targettypes, [etype]
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   517
                self._synchronize_rschema(rschema, syncrdefs=False,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   518
                                          syncprops=syncprops, syncperms=syncperms)
10284
fb113f9fa7d8 [migration] sync_schema_props_perms should skip computed relations. Closes #5147796
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10129
diff changeset
   519
                if rschema.rule: # rdef for computed rtype are infered hence should not be
fb113f9fa7d8 [migration] sync_schema_props_perms should skip computed relations. Closes #5147796
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10129
diff changeset
   520
                                 # synchronized
fb113f9fa7d8 [migration] sync_schema_props_perms should skip computed relations. Closes #5147796
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10129
diff changeset
   521
                    continue
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   522
                reporschema = self.repo.schema.rschema(rschema)
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   523
                for subj in subjtypes:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   524
                    for obj in objtypes:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   525
                        if (subj, obj) not in reporschema.rdefs:
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   526
                            continue
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   527
                        self._synchronize_rdef_schema(subj, rschema, obj,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
   528
                                                      syncprops=syncprops, syncperms=syncperms)
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6205
diff changeset
   529
        if syncprops: # need to process __unique_together__ after rdefs were processed
9375
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   530
            # mappings from constraint name to columns
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   531
            # filesystem (fs) and repository (repo) wise
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   532
            fs = {}
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   533
            repo = {}
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   534
            for cols in eschema._unique_together or ():
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   535
                fs[unique_index_name(repoeschema, cols)] = sorted(cols)
9556
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   536
            schemaentity = self.cnx.entity_from_eid(repoeschema.eid)
9375
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   537
            for entity in schemaentity.related('constraint_of', 'object',
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   538
                                               targettypes=('CWUniqueTogetherConstraint',)).entities():
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   539
                repo[entity.name] = sorted(rel.name for rel in entity.relations)
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   540
            added = set(fs) - set(repo)
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   541
            removed = set(repo) - set(fs)
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   542
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   543
            for name in removed:
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   544
                self.rqlexec('DELETE CWUniqueTogetherConstraint C WHERE C name %(name)s',
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   545
                             {'name': name})
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   546
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   547
            def possible_unique_constraint(cols):
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   548
                for name in cols:
9195
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   549
                    rschema = repoeschema.subjrels.get(name)
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   550
                    if rschema is None:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   551
                        print('dont add %s unique constraint on %s, missing %s' % (
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   552
                            ','.join(cols), eschema, name))
9195
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   553
                        return False
9241
cbee712dd310 [migractions] rschema.final.inlined -> rschema.inlined
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9195
diff changeset
   554
                    if not (rschema.final or rschema.inlined):
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   555
                        print('dont add %s unique constraint on %s, %s is neither final nor inlined' % (
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   556
                            ','.join(cols), eschema, name))
9195
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   557
                        return False
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   558
                return True
9375
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   559
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   560
            for name in added:
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   561
                if possible_unique_constraint(fs[name]):
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   562
                    rql, substs = ss._uniquetogether2rql(eschema, fs[name])
9195
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   563
                    substs['x'] = repoeschema.eid
9375
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   564
                    substs['name'] = name
9195
e9fb9377229e [migration.sync_schema_props_perms] ensure all participants to a unique together constraint are there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9175
diff changeset
   565
                    self.rqlexec(rql, substs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   566
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   567
    def _synchronize_rdef_schema(self, subjtype, rtype, objtype,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   568
                                 syncperms=True, syncprops=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   569
        """synchronize properties of the persistent relation definition schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   570
        against its current definition:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   571
        * order and other properties
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   572
        * constraints
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   573
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   574
        subjtype, objtype = str(subjtype), str(objtype)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   575
        rschema = self.fs_schema.rschema(rtype)
9964
f4a3ee05cf9d [CWEP002 migration] support sync_schema_props_perms for computed relations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9963
diff changeset
   576
        if rschema.rule:
f4a3ee05cf9d [CWEP002 migration] support sync_schema_props_perms for computed relations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9963
diff changeset
   577
            raise ExecutionError('Cannot synchronize a relation definition for a '
f4a3ee05cf9d [CWEP002 migration] support sync_schema_props_perms for computed relations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9963
diff changeset
   578
                                 'computed relation (%s)' % rschema)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   579
        reporschema = self.repo.schema.rschema(rschema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   580
        if (subjtype, rschema, objtype) in self._synchronized:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   581
            return
10584
743ed2b13a6f [syncschema] only add to the `synchronized` set if all possible updates have been done
Aurelien Campeas <aurelien.campeas@pythonian.fr>
parents: 9773
diff changeset
   582
        if syncperms and syncprops:
743ed2b13a6f [syncschema] only add to the `synchronized` set if all possible updates have been done
Aurelien Campeas <aurelien.campeas@pythonian.fr>
parents: 9773
diff changeset
   583
            self._synchronized.add((subjtype, rschema, objtype))
743ed2b13a6f [syncschema] only add to the `synchronized` set if all possible updates have been done
Aurelien Campeas <aurelien.campeas@pythonian.fr>
parents: 9773
diff changeset
   584
            if rschema.symmetric:
743ed2b13a6f [syncschema] only add to the `synchronized` set if all possible updates have been done
Aurelien Campeas <aurelien.campeas@pythonian.fr>
parents: 9773
diff changeset
   585
                self._synchronized.add((objtype, rschema, subjtype))
4949
f4dce73da26b [schema sync] don't try to synchronize infered relation defs, fixing a name error on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4947
diff changeset
   586
        rdef = rschema.rdef(subjtype, objtype)
f4dce73da26b [schema sync] don't try to synchronize infered relation defs, fixing a name error on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4947
diff changeset
   587
        if rdef.infered:
f4dce73da26b [schema sync] don't try to synchronize infered relation defs, fixing a name error on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4947
diff changeset
   588
            return # don't try to synchronize infered relation defs
4950
bca0873c0d6e [schema sync] fix another potential name error on synchronizing rdefs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4949
diff changeset
   589
        repordef = reporschema.rdef(subjtype, objtype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   590
        confirm = self.verbosity >= 2
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   591
        if syncprops:
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   592
            # properties
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   593
            self.rqlexecall(ss.updaterdef2rql(rdef, repordef.eid),
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   594
                            ask_confirm=confirm)
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   595
            # constraints
9565
fa00fc251d57 [migractions] Better handle removal of RQLConstraint in sync_schema
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9241
diff changeset
   596
            # 0. eliminate the set of unmodified constraints from the sets of
fa00fc251d57 [migractions] Better handle removal of RQLConstraint in sync_schema
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9241
diff changeset
   597
            # old/new constraints
fa00fc251d57 [migractions] Better handle removal of RQLConstraint in sync_schema
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9241
diff changeset
   598
            newconstraints = set(rdef.constraints)
fa00fc251d57 [migractions] Better handle removal of RQLConstraint in sync_schema
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9241
diff changeset
   599
            oldconstraints = set(repordef.constraints)
fa00fc251d57 [migractions] Better handle removal of RQLConstraint in sync_schema
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9241
diff changeset
   600
            unchanged_constraints = newconstraints & oldconstraints
fa00fc251d57 [migractions] Better handle removal of RQLConstraint in sync_schema
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9241
diff changeset
   601
            newconstraints -= unchanged_constraints
fa00fc251d57 [migractions] Better handle removal of RQLConstraint in sync_schema
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9241
diff changeset
   602
            oldconstraints -= unchanged_constraints
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   603
            # 1. remove old constraints and update constraints of the same type
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   604
            # NOTE: don't use rschema.constraint_by_type because it may be
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   605
            #       out of sync with newconstraints when multiple
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   606
            #       constraints of the same type are used
9565
fa00fc251d57 [migractions] Better handle removal of RQLConstraint in sync_schema
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9241
diff changeset
   607
            for cstr in oldconstraints:
9636
e35ae8617c03 Fix constraint sync during migration
Julien Cristau <julien.cristau@logilab.fr>
parents: 9635
diff changeset
   608
                self.rqlexec('DELETE CWConstraint C WHERE C eid %(x)s',
e35ae8617c03 Fix constraint sync during migration
Julien Cristau <julien.cristau@logilab.fr>
parents: 9635
diff changeset
   609
                             {'x': cstr.eid}, ask_confirm=confirm)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
   610
            # 2. add new constraints
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   611
            cstrtype_map = self.cstrtype_mapping()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   612
            self.rqlexecall(ss.constraints2rql(cstrtype_map, newconstraints,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   613
                                               repordef.eid),
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   614
                            ask_confirm=confirm)
4188
b3258d2afe04 fix virtual relation detection: erschema is no more a relation type but a relation def in _synchronize_permissions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4099
diff changeset
   615
        if syncperms and not rschema in VIRTUAL_RTYPES:
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
   616
            self._synchronize_permissions(rdef, repordef.eid)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   617
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   618
    # base actions ############################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   619
4011
394f853bb653 [migration] write migration instructions for permissions handling on relation definition
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3998
diff changeset
   620
    def checkpoint(self, ask_confirm=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   621
        """checkpoint action"""
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
   622
        if not ask_confirm or self.confirm('Commit now ?', shell=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   623
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   624
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   625
    def cmd_add_cube(self, cube, update_database=True):
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   626
        self.cmd_add_cubes( (cube,), update_database)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   627
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   628
    def cmd_add_cubes(self, cubes, update_database=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   629
        """update_database is telling if the database schema should be updated
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   630
        or if only the relevant eproperty should be inserted (for the case where
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2275
diff changeset
   631
        a cube has been extracted from an existing instance, so the
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   632
        cube schema is already in there)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   633
        """
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   634
        newcubes = super(ServerMigrationHelper, self).cmd_add_cubes(cubes)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   635
        if not newcubes:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   636
            return
2107
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   637
        for cube in newcubes:
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   638
            self.cmd_set_property('system.version.'+cube,
6c4a4c514ac2 add source to sources configuration when adding a cube defining a source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   639
                                  self.config.cube_version(cube))
3836
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   640
            # ensure added cube is in config cubes
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   641
            # XXX worth restoring on error?
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   642
            if not cube in self.config._cubes:
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   643
                self.config._cubes += (cube,)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   644
        if not update_database:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   645
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   646
            return
1036
593df4919845 when reading the schema while adding/removing cubes, read schema in non-strict mode
sylvain.thenault@logilab.fr
parents: 1016
diff changeset
   647
        newcubes_schema = self.config.load_schema(construction_mode='non-strict')
3836
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   648
        # XXX we have to replace fs_schema, used in cmd_add_relation_type
58c09f21f503 fix necessary when adding a cube which isn't listed in fs dependencies of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3715
diff changeset
   649
        # etc. and fsschema of migration script contexts
6217
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6208
diff changeset
   650
        self.fs_schema = newcubes_schema
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6208
diff changeset
   651
        self.update_context('fsschema', self.fs_schema)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   652
        new = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   653
        # execute pre-create files
3903
7967d3766ecf fix #544758 by calling custom sql scripts in add_cubes.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3872
diff changeset
   654
        driver = self.repo.system_source.dbdriver
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   655
        for cube in reversed(newcubes):
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   656
            self.cmd_install_custom_sql_scripts(cube)
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   657
            self.cmd_exec_event_script('precreate', cube)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   658
        # add new entity and relation types
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   659
        for rschema in newcubes_schema.relations():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   660
            if not rschema in self.repo.schema:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   661
                self.cmd_add_relation_type(rschema.type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   662
                new.add(rschema.type)
2926
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   663
        toadd = [eschema for eschema in newcubes_schema.entities()
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   664
                 if not eschema in self.repo.schema]
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   665
        for eschema in order_eschemas(toadd):
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   666
            self.cmd_add_entity_type(eschema.type)
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   667
            new.add(eschema.type)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   668
        # check if attributes has been added to existing entities
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   669
        for rschema in newcubes_schema.relations():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   670
            existingschema = self.repo.schema.rschema(rschema.type)
4099
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   671
            for (fromtype, totype) in rschema.rdefs:
9164
4415209bea0a [migration] when adding a cube, skip infered relations (closes #3005576)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9162
diff changeset
   672
                # if rdef already exists or is infered from inheritance,
4415209bea0a [migration] when adding a cube, skip infered relations (closes #3005576)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9162
diff changeset
   673
                # don't add it
4415209bea0a [migration] when adding a cube, skip infered relations (closes #3005576)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9162
diff changeset
   674
                if (fromtype, totype) in existingschema.rdefs \
4415209bea0a [migration] when adding a cube, skip infered relations (closes #3005576)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9162
diff changeset
   675
                        or rschema.rdefs[(fromtype, totype)].infered:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   676
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   677
                # check we should actually add the relation definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   678
                if not (fromtype in new or totype in new or rschema in new):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   679
                    continue
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   680
                self.cmd_add_relation_definition(str(fromtype), rschema.type,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   681
                                                 str(totype))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   682
        # execute post-create files
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   683
        for cube in reversed(newcubes):
9556
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   684
            with self.cnx.allow_all_hooks_but():
8210
1d1cfc97f6b9 [migration] fire hooks in postcreate by default (closes #1417110)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8190
diff changeset
   685
                self.cmd_exec_event_script('postcreate', cube)
1d1cfc97f6b9 [migration] fire hooks in postcreate by default (closes #1417110)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8190
diff changeset
   686
                self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   687
10107
1ef92a6193a8 [serverctl] rename remove_cube to drop_cube (closes #4545093)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 10095
diff changeset
   688
    def cmd_drop_cube(self, cube, removedeps=False):
1ef92a6193a8 [serverctl] rename remove_cube to drop_cube (closes #4545093)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 10095
diff changeset
   689
        removedcubes = super(ServerMigrationHelper, self).cmd_drop_cube(
2122
4ea13a828513 add removedeps option to remove_cube to control wether a cube's dependencies should be removed as well or not
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2107
diff changeset
   690
            cube, removedeps)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   691
        if not removedcubes:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   692
            return
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   693
        fsschema = self.fs_schema
1036
593df4919845 when reading the schema while adding/removing cubes, read schema in non-strict mode
sylvain.thenault@logilab.fr
parents: 1016
diff changeset
   694
        removedcubes_schema = self.config.load_schema(construction_mode='non-strict')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   695
        reposchema = self.repo.schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   696
        # execute pre-remove files
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   697
        for cube in reversed(removedcubes):
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   698
            self.cmd_exec_event_script('preremove', cube)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   699
        # remove cubes'entity and relation types
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   700
        for rschema in fsschema.relations():
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   701
            if not rschema in removedcubes_schema and rschema in reposchema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   702
                self.cmd_drop_relation_type(rschema.type)
2926
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   703
        toremove = [eschema for eschema in fsschema.entities()
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   704
                    if not eschema in removedcubes_schema
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   705
                    and eschema in reposchema]
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   706
        for eschema in reversed(order_eschemas(toremove)):
4484387ed012 when adding/removing cubes, we should add/remove entity types in correct order if one inherits from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
   707
            self.cmd_drop_entity_type(eschema.type)
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   708
        for rschema in fsschema.relations():
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   709
            if rschema in removedcubes_schema and rschema in reposchema:
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   710
                # check if attributes/relations has been added to entities from
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   711
                # other cubes
4099
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   712
                for fromtype, totype in rschema.rdefs:
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   713
                    if (fromtype, totype) not in removedcubes_schema[rschema.type].rdefs and \
59ff385f7348 yams api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4043
diff changeset
   714
                           (fromtype, totype) in reposchema[rschema.type].rdefs:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   715
                        self.cmd_drop_relation_definition(
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   716
                            str(fromtype), rschema.type, str(totype))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   717
        # execute post-remove files
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   718
        for cube in reversed(removedcubes):
7915
a7f3245e1728 [migration] expose migration methods to execute schema/*.sql and migration/<event>.py files (closes #1986498)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
   719
            self.cmd_exec_event_script('postremove', cube)
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   720
            self.rqlexec('DELETE CWProperty X WHERE X pkey %(pk)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   721
                         {'pk': u'system.version.'+cube}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   722
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   723
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   724
    # schema migration actions ################################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   725
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   726
    def cmd_add_attribute(self, etype, attrname, attrtype=None, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   727
        """add a new attribute on the given entity type"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   728
        if attrtype is None:
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   729
            rschema = self.fs_schema.rschema(attrname)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   730
            attrtype = rschema.objects(etype)[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   731
        self.cmd_add_relation_definition(etype, attrname, attrtype, commit=commit)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   732
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   733
    def cmd_drop_attribute(self, etype, attrname, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   734
        """drop an existing attribute from the given entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   735
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   736
        `attrname` is a string giving the name of the attribute to drop
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   737
        """
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   738
        try:
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   739
            rschema = self.repo.schema.rschema(attrname)
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   740
            attrtype = rschema.objects(etype)[0]
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   741
        except KeyError:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   742
            print('warning: attribute %s %s is not known, skip deletion' % (
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   743
                etype, attrname))
7558
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   744
        else:
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   745
            self.cmd_drop_relation_definition(etype, attrname, attrtype,
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   746
                                              commit=commit)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   747
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   748
    def cmd_rename_attribute(self, etype, oldname, newname, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   749
        """rename an existing attribute of the given entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   750
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   751
        `oldname` is a string giving the name of the existing attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   752
        `newname` is a string giving the name of the renamed attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   753
        """
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
   754
        eschema = self.fs_schema.eschema(etype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   755
        attrtype = eschema.destination(newname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   756
        # have to commit this first step anyway to get the definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   757
        # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   758
        self.cmd_add_attribute(etype, newname, attrtype, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   759
        # skipp NULL values if the attribute is required
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   760
        rql = 'SET X %s VAL WHERE X is %s, X %s VAL' % (newname, etype, oldname)
4633
a85b4361fb22 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4580
diff changeset
   761
        card = eschema.rdef(newname).cardinality[0]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   762
        if card == '1':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   763
            rql += ', NOT X %s NULL' % oldname
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   764
        self.rqlexec(rql, ask_confirm=self.verbosity>=2)
3548
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   765
        # XXX if both attributes fulltext indexed, should skip fti rebuild
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   766
        # XXX if old attribute was fti indexed but not the new one old value
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   767
        # won't be removed from the index (this occurs on other kind of
4cf5a360952e add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3213
diff changeset
   768
        # fulltextindexed change...)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   769
        self.cmd_drop_attribute(etype, oldname, commit=commit)
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   770
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   771
    def cmd_add_entity_type(self, etype, auto=True, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   772
        """register a new entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   773
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   774
        in auto mode, automatically register entity's relation where the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   775
        targeted type is known
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   776
        """
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   777
        instschema = self.repo.schema
7166
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7161
diff changeset
   778
        eschema = self.fs_schema.eschema(etype)
7558
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
   779
        if etype in instschema and not (eschema.final and eschema.eid is None):
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   780
            print('warning: %s already known, skip addition' % etype)
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
   781
            return
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   782
        confirm = self.verbosity >= 2
4043
39ae94e0c8b8 give group mapping where needed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4042
diff changeset
   783
        groupmap = self.group_mapping()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   784
        cstrtypemap = self.cstrtype_mapping()
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   785
        # register the entity into CWEType
9511
241b1232ed7f Use repoapi instead of dbapi for cwctl shell, upgrade and db-init
Julien Cristau <julien.cristau@logilab.fr>
parents: 9478
diff changeset
   786
        execute = self.cnx.execute
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   787
        ss.execschemarql(execute, eschema, ss.eschema2rql(eschema, groupmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   788
        # add specializes relation if needed
5293
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   789
        specialized = eschema.specializes()
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   790
        if specialized:
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   791
            try:
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   792
                specialized.eid = instschema[specialized].eid
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   793
            except KeyError:
6323
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   794
                raise ExecutionError('trying to add entity type but parent type is '
a11c1e3c16c3 [c-c shell/upgrade] raise ExecutionError when traceback is not relevant
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6217
diff changeset
   795
                                     'not yet in the database schema')
5293
72e102a06709 [migration] fix bug when trying to add a new entity type that inherit from another
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5289
diff changeset
   796
            self.rqlexecall(ss.eschemaspecialize2rql(eschema), ask_confirm=confirm)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   797
        # register entity's attributes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   798
        for rschema, attrschema in eschema.attribute_definitions():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   799
            # ignore those meta relations, they will be automatically added
2617
89c62b855f2e [R schema hooks] rename META_RELATIONS_TYPES into META_RTYPES, use it and other schema consts to avoid errors
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2616
diff changeset
   800
            if rschema.type in META_RTYPES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   801
                continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   802
            if not rschema.type in instschema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   803
                # need to add the relation type and to commit to get it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   804
                # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   805
                self.cmd_add_relation_type(rschema.type, False, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   806
            # register relation definition
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   807
            rdef = self._get_rdef(rschema, eschema, eschema.destination(rschema))
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   808
            ss.execschemarql(execute, rdef, ss.rdef2rql(rdef, cstrtypemap, groupmap),)
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   809
        # take care to newly introduced base class
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   810
        # XXX some part of this should probably be under the "if auto" block
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   811
        for spschema in eschema.specialized_by(recursive=False):
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   812
            try:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   813
                instspschema = instschema[spschema]
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   814
            except KeyError:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   815
                # specialized entity type not in schema, ignore
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   816
                continue
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   817
            if instspschema.specializes() != eschema:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   818
                self.rqlexec('SET D specializes P WHERE D eid %(d)s, P name %(pn)s',
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   819
                             {'d': instspschema.eid, 'pn': eschema.type},
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
   820
                             ask_confirm=confirm)
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   821
                for rschema, tschemas, role in spschema.relation_definitions(True):
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   822
                    for tschema in tschemas:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   823
                        if not tschema in instschema:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   824
                            continue
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   825
                        if role == 'subject':
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   826
                            subjschema = spschema
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   827
                            objschema = tschema
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3582
diff changeset
   828
                            if rschema.final and rschema in instspschema.subjrels:
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   829
                                # attribute already set, has_rdef would check if
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   830
                                # it's of the same type, we don't want this so
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   831
                                # simply skip here
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   832
                                continue
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   833
                        elif role == 'object':
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   834
                            subjschema = tschema
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   835
                            objschema = spschema
4633
a85b4361fb22 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4580
diff changeset
   836
                        if (rschema.rdef(subjschema, objschema).infered
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   837
                            or (instschema.has_relation(rschema) and
4633
a85b4361fb22 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4580
diff changeset
   838
                                (subjschema, objschema) in instschema[rschema].rdefs)):
4721
8f63691ccb7f pylint style fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   839
                            continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   840
                        self.cmd_add_relation_definition(
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   841
                            subjschema.type, rschema.type, objschema.type)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   842
        if auto:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   843
            # we have commit here to get relation types actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   844
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   845
            added = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   846
            for rschema in eschema.subject_relations():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   847
                # attribute relation have already been processed and
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   848
                # 'owned_by'/'created_by' will be automatically added
2617
89c62b855f2e [R schema hooks] rename META_RELATIONS_TYPES into META_RTYPES, use it and other schema consts to avoid errors
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2616
diff changeset
   849
                if rschema.final or rschema.type in META_RTYPES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   850
                    continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   851
                rtypeadded = rschema.type in instschema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   852
                for targetschema in rschema.objects(etype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   853
                    # ignore relations where the targeted type is not in the
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2275
diff changeset
   854
                    # current instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   855
                    targettype = targetschema.type
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   856
                    if not targettype in instschema and targettype != etype:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   857
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   858
                    if not rtypeadded:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   859
                        # need to add the relation type and to commit to get it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   860
                        # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   861
                        added.append(rschema.type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   862
                        self.cmd_add_relation_type(rschema.type, False, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   863
                        rtypeadded = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   864
                    # register relation definition
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4367
diff changeset
   865
                    # remember this two avoid adding twice non symmetric relation
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   866
                    # such as "Emailthread forked_from Emailthread"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   867
                    added.append((etype, rschema.type, targettype))
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   868
                    rdef = self._get_rdef(rschema, eschema, targetschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   869
                    ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   870
                                     ss.rdef2rql(rdef, cstrtypemap, groupmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   871
            for rschema in eschema.object_relations():
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   872
                if rschema.type in META_RTYPES:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   873
                    continue
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   874
                rtypeadded = rschema.type in instschema or rschema.type in added
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   875
                for targetschema in rschema.subjects(etype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   876
                    # ignore relations where the targeted type is not in the
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2275
diff changeset
   877
                    # current instance schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   878
                    targettype = targetschema.type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   879
                    # don't check targettype != etype since in this case the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   880
                    # relation has already been added as a subject relation
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2962
diff changeset
   881
                    if not targettype in instschema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   882
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   883
                    if not rtypeadded:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   884
                        # need to add the relation type and to commit to get it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   885
                        # actually in the schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   886
                        self.cmd_add_relation_type(rschema.type, False, commit=True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   887
                        rtypeadded = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   888
                    elif (targettype, rschema.type, etype) in added:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   889
                        continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   890
                    # register relation definition
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   891
                    rdef = self._get_rdef(rschema, targetschema, eschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   892
                    ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   893
                                     ss.rdef2rql(rdef, cstrtypemap, groupmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   894
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   895
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   896
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   897
    def cmd_drop_entity_type(self, etype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   898
        """unregister an existing entity type
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   899
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   900
        This will trigger deletion of necessary relation types and definitions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   901
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   902
        # XXX what if we delete an entity type which is specialized by other types
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   903
        # unregister the entity from CWEType
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
   904
        self.rqlexec('DELETE CWEType X WHERE X name %(etype)s', {'etype': etype},
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   905
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   906
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   907
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   908
6115
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   909
    def cmd_rename_entity_type(self, oldname, newname, attrs=None, commit=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   910
        """rename an existing entity type in the persistent schema
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
   911
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   912
        `oldname` is a string giving the name of the existing entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   913
        `newname` is a string giving the name of the renamed entity type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   914
        """
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   915
        schema = self.repo.schema
9162
bf0a575d8da8 [migraction] rename_entity_type simply warn if old entity type isn't in the schema. Closes #3004069
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8807
diff changeset
   916
        if oldname not in schema:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   917
            print('warning: entity type %s is unknown, skip renaming' % oldname)
9162
bf0a575d8da8 [migraction] rename_entity_type simply warn if old entity type isn't in the schema. Closes #3004069
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8807
diff changeset
   918
            return
bf0a575d8da8 [migraction] rename_entity_type simply warn if old entity type isn't in the schema. Closes #3004069
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8807
diff changeset
   919
        # if merging two existing entity types
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   920
        if newname in schema:
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   921
            assert oldname in ETYPE_NAME_MAP, \
6115
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   922
                   '%s should be mapped to %s in ETYPE_NAME_MAP' % (oldname,
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   923
                                                                    newname)
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   924
            if attrs is None:
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   925
                attrs = ','.join(SQL_PREFIX + rschema.type
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   926
                                 for rschema in schema[newname].subject_relations()
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   927
                                 if (rschema.final or rschema.inlined)
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   928
                                 and not rschema in PURE_VIRTUAL_RTYPES)
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   929
            else:
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   930
                attrs += ('eid', 'creation_date', 'modification_date', 'cwuri')
56ec278b843e [migration] we can now specify attributes to backport to rename_entity_type using its attrs argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6075
diff changeset
   931
                attrs = ','.join(SQL_PREFIX + attr for attr in attrs)
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   932
            self.sqlexec('INSERT INTO %s%s(%s) SELECT %s FROM %s%s' % (
5999
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   933
                SQL_PREFIX, newname, attrs, attrs, SQL_PREFIX, oldname),
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   934
                         ask_confirm=False)
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   935
            # old entity type has not been added to the schema, can't gather it
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   936
            new = schema.eschema(newname)
6117
f3fdfcbda54b cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6116
diff changeset
   937
            oldeid = self.rqlexec('CWEType ET WHERE ET name %(on)s',
f3fdfcbda54b cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6116
diff changeset
   938
                                  {'on': oldname}, ask_confirm=False)[0][0]
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   939
            # backport old type relations to new type
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   940
            # XXX workflows, other relations?
5999
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   941
            for r1, rr1 in [('from_entity', 'to_entity'),
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   942
                            ('to_entity', 'from_entity')]:
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   943
                self.rqlexec('SET X %(r1)s NET WHERE X %(r1)s OET, '
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   944
                             'NOT EXISTS(X2 %(r1)s NET, X relation_type XRT, '
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   945
                             'X2 relation_type XRT, X %(rr1)s XTE, X2 %(rr1)s XTE), '
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   946
                             'OET eid %%(o)s, NET eid %%(n)s' % locals(),
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   947
                             {'o': oldeid, 'n': new.eid}, ask_confirm=False)
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   948
            # backport is / is_instance_of relation to new type
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   949
            for rtype in ('is', 'is_instance_of'):
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   950
                self.sqlexec('UPDATE %s_relation SET eid_to=%s WHERE eid_to=%s'
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   951
                             % (rtype, new.eid, oldeid), ask_confirm=False)
eaf8219f8b7d [migration] fix rename_entity_type to avoid to loose some relations on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5928
diff changeset
   952
            # delete relations using SQL to avoid relations content removal
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   953
            # triggered by schema synchronization hooks.
6116
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   954
            for rdeftype in ('CWRelation', 'CWAttribute'):
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   955
                thispending = set( (eid for eid, in self.sqlexec(
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   956
                    'SELECT cw_eid FROM cw_%s WHERE cw_from_entity=%%(eid)s OR '
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   957
                    ' cw_to_entity=%%(eid)s' % rdeftype,
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   958
                    {'eid': oldeid}, ask_confirm=False)) )
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   959
                # we should add deleted eids into pending eids else we may
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   960
                # get some validation error on commit since integrity hooks
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   961
                # may think some required relation is missing... This also ensure
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   962
                # repository caches are properly cleanup
10353
d9a1e7939ee6 [migractions] remove any session related leftovers
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10346
diff changeset
   963
                hook.CleanupDeletedEidsCacheOp.get_instance(self.cnx).union(thispending)
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   964
                # and don't forget to remove record from system tables
9556
12ee310541bb [server/migractions] finish migration to repoapi objects
Julien Cristau <julien.cristau@logilab.fr>
parents: 9555
diff changeset
   965
                entities = [self.cnx.entity_from_eid(eid, rdeftype) for eid in thispending]
10353
d9a1e7939ee6 [migractions] remove any session related leftovers
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10346
diff changeset
   966
                self.repo.system_source.delete_info_multi(self.cnx, entities)
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7410
diff changeset
   967
                self.sqlexec('DELETE FROM cw_%s WHERE cw_from_entity=%%(eid)s OR '
6116
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   968
                             'cw_to_entity=%%(eid)s' % rdeftype,
2530e9c45296 [migration] fix rename_entity_type: avoid removal of attributes on the new entity type from the schema during migration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6115
diff changeset
   969
                             {'eid': oldeid}, ask_confirm=False)
6190
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   970
                # now we have to manually cleanup relations pointing to deleted
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   971
                # entities
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   972
                thiseids = ','.join(str(eid) for eid in thispending)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   973
                for rschema, ttypes, role in schema[rdeftype].relation_definitions():
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   974
                    if rschema.type in VIRTUAL_RTYPES:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   975
                        continue
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   976
                    sqls = []
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   977
                    if role == 'object':
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   978
                        if rschema.inlined:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   979
                            for eschema in ttypes:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   980
                                sqls.append('DELETE FROM cw_%s WHERE cw_%s IN(%%s)'
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   981
                                            % (eschema, rschema))
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   982
                        else:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   983
                            sqls.append('DELETE FROM %s_relation WHERE eid_to IN(%%s)'
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   984
                                        % rschema)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   985
                    elif not rschema.inlined:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   986
                        sqls.append('DELETE FROM %s_relation WHERE eid_from IN(%%s)'
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   987
                                    % rschema)
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   988
                    for sql in sqls:
564a6028067c [migration] more rename_entity_type fix
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   989
                        self.sqlexec(sql % thiseids, ask_confirm=False)
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   990
            # remove the old type: use rql to propagate deletion
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   991
            self.rqlexec('DELETE CWEType ET WHERE ET name %(on)s', {'on': oldname},
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   992
                         ask_confirm=False)
9162
bf0a575d8da8 [migraction] rename_entity_type simply warn if old entity type isn't in the schema. Closes #3004069
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8807
diff changeset
   993
        # elif simply renaming an entity type
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   994
        else:
5897
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   995
            self.rqlexec('SET ET name %(newname)s WHERE ET is CWEType, ET name %(on)s',
0c6f2f866202 [migration] fix rename_entity_type to an existant entity type: get back relation from the old entity type non existant on the target type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5863
diff changeset
   996
                         {'newname' : unicode(newname), 'on' : oldname},
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   997
                         ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   998
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   999
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1000
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1001
    def cmd_add_relation_type(self, rtype, addrdef=True, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1002
        """register a new relation type named `rtype`, as described in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1003
        schema description file.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1004
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1005
        `addrdef` is a boolean value; when True, it will also add all relations
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1006
        of the type just added found in the schema definition file. Note that it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1007
        implies an intermediate "commit" which commits the relation type
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1008
        creation (but not the relation definitions themselves, for which
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1009
        committing depends on the `commit` argument value).
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1010
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1011
        """
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1012
        reposchema = self.repo.schema
8209
67660d5f1916 [migration] `add_relation_type` warns and add missing definitions on already existing rtypes (closes #2121979)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8049
diff changeset
  1013
        rschema = self.fs_schema.rschema(rtype)
9511
241b1232ed7f Use repoapi instead of dbapi for cwctl shell, upgrade and db-init
Julien Cristau <julien.cristau@logilab.fr>
parents: 9478
diff changeset
  1014
        execute = self.cnx.execute
7558
044135305359 [migration] litle enhancements and a fix to patch accidentally pushed two revisions ago
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7556
diff changeset
  1015
        if rtype in reposchema:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
  1016
            print('warning: relation type %s is already known, skip addition' % (
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
  1017
                rtype))
9963
5531f5577b50 [CWEP002 migration] support add_relation_type for computed relations
Laura Médioni <laura.medioni@logilab.fr>
parents: 9962
diff changeset
  1018
        elif rschema.rule:
10553
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10473
diff changeset
  1019
            gmap = self.group_mapping()
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10473
diff changeset
  1020
            ss.execschemarql(execute, rschema, ss.crschema2rql(rschema, gmap))
8209
67660d5f1916 [migration] `add_relation_type` warns and add missing definitions on already existing rtypes (closes #2121979)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8049
diff changeset
  1021
        else:
67660d5f1916 [migration] `add_relation_type` warns and add missing definitions on already existing rtypes (closes #2121979)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8049
diff changeset
  1022
            # register the relation into CWRType and insert necessary relation
67660d5f1916 [migration] `add_relation_type` warns and add missing definitions on already existing rtypes (closes #2121979)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8049
diff changeset
  1023
            # definitions
67660d5f1916 [migration] `add_relation_type` warns and add missing definitions on already existing rtypes (closes #2121979)
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 8049
diff changeset
  1024
            ss.execschemarql(execute, rschema, ss.rschema2rql(rschema, addrdef=False))
9963
5531f5577b50 [CWEP002 migration] support add_relation_type for computed relations
Laura Médioni <laura.medioni@logilab.fr>
parents: 9962
diff changeset
  1025
        if not rschema.rule and addrdef:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1026
            self.commit()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1027
            gmap = self.group_mapping()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1028
            cmap = self.cstrtype_mapping()
6073
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1029
            done = set()
8285
32098d98bf2f [migration] fix add_relation_type in case it already exists, broken by 67660d5f1916
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8278
diff changeset
  1030
            for subj, obj in rschema.rdefs:
32098d98bf2f [migration] fix add_relation_type in case it already exists, broken by 67660d5f1916
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8278
diff changeset
  1031
                if not (reposchema.has_entity(subj)
32098d98bf2f [migration] fix add_relation_type in case it already exists, broken by 67660d5f1916
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8278
diff changeset
  1032
                        and reposchema.has_entity(obj)):
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1033
                    continue
6073
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1034
                # symmetric relations appears twice
8285
32098d98bf2f [migration] fix add_relation_type in case it already exists, broken by 67660d5f1916
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8278
diff changeset
  1035
                if (subj, obj) in done:
6073
bf684f1022bf [migration] fix crash when adding symmetric relation type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
  1036
                    continue
8285
32098d98bf2f [migration] fix add_relation_type in case it already exists, broken by 67660d5f1916
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8278
diff changeset
  1037
                done.add( (subj, obj) )
32098d98bf2f [migration] fix add_relation_type in case it already exists, broken by 67660d5f1916
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8278
diff changeset
  1038
                self.cmd_add_relation_definition(subj, rtype, obj)
2700
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1039
            if rtype in META_RTYPES:
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1040
                # if the relation is in META_RTYPES, ensure we're adding it for
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1041
                # all entity types *in the persistent schema*, not only those in
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1042
                # the fs schema
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1043
                for etype in self.repo.schema.entities():
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1044
                    if not etype in self.fs_schema:
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1045
                        # get sample object type and rproperties
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1046
                        objtypes = rschema.objects()
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1047
                        assert len(objtypes) == 1, objtypes
2700
ecf888c8a250 [migration] when adding a core meta-relation, it should be added to all entities *in the persistent schema*, not only those in the fs schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2680
diff changeset
  1048
                        objtype = objtypes[0]
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1049
                        rdef = copy(rschema.rdef(rschema.subjects(objtype)[0], objtype))
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1050
                        rdef.subject = etype
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1051
                        rdef.rtype = self.repo.schema.rschema(rschema)
8616
820cb9cee702 [migration] fix crash when adding a new meta relation type. Closes #2532483
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8530
diff changeset
  1052
                        rdef.object = self.repo.schema.eschema(objtype)
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1053
                        ss.execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1054
                                         ss.rdef2rql(rdef, cmap, gmap))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1055
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1056
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1057
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1058
    def cmd_drop_relation_type(self, rtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1059
        """unregister an existing relation type"""
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1060
        self.rqlexec('DELETE CWRType X WHERE X name %r' % rtype,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1061
                     ask_confirm=self.verbosity>=2)
10129
52adf66fae7d [migration] don't crash when deleting a non-existing rtype
Julien Cristau <julien.cristau@logilab.fr>
parents: 10107
diff changeset
  1062
        self.rqlexec('DELETE CWComputedRType X WHERE X name %r' % rtype,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1063
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1064
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1065
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1066
9698
737983d87497 [migractions] add sanity check in rename_relation_type
Julien Cristau <julien.cristau@logilab.fr>
parents: 9600
diff changeset
  1067
    def cmd_rename_relation_type(self, oldname, newname, commit=True, force=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1068
        """rename an existing relation
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1069
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1070
        `oldname` is a string giving the name of the existing relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1071
        `newname` is a string giving the name of the renamed relation
9698
737983d87497 [migractions] add sanity check in rename_relation_type
Julien Cristau <julien.cristau@logilab.fr>
parents: 9600
diff changeset
  1072
737983d87497 [migractions] add sanity check in rename_relation_type
Julien Cristau <julien.cristau@logilab.fr>
parents: 9600
diff changeset
  1073
        If `force` is True, proceed even if `oldname` still appears in the fs schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1074
        """
9698
737983d87497 [migractions] add sanity check in rename_relation_type
Julien Cristau <julien.cristau@logilab.fr>
parents: 9600
diff changeset
  1075
        if oldname in self.fs_schema and not force:
737983d87497 [migractions] add sanity check in rename_relation_type
Julien Cristau <julien.cristau@logilab.fr>
parents: 9600
diff changeset
  1076
            if not self.confirm('Relation %s is still present in the filesystem schema,'
737983d87497 [migractions] add sanity check in rename_relation_type
Julien Cristau <julien.cristau@logilab.fr>
parents: 9600
diff changeset
  1077
                                ' do you really want to drop it?' % oldname,
737983d87497 [migractions] add sanity check in rename_relation_type
Julien Cristau <julien.cristau@logilab.fr>
parents: 9600
diff changeset
  1078
                                default='n'):
10571
2cd8933fd875 [server/migractions] raising SystemExit sounds like a bad idea
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10553
diff changeset
  1079
                return
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1080
        self.cmd_add_relation_type(newname, commit=True)
10632
d89e9176d263 [migration] don't attempt to carry over values when renaming a computed relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10284
diff changeset
  1081
        if not self.repo.schema[oldname].rule:
d89e9176d263 [migration] don't attempt to carry over values when renaming a computed relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10284
diff changeset
  1082
            self.rqlexec('SET X %s Y WHERE X %s Y' % (newname, oldname),
d89e9176d263 [migration] don't attempt to carry over values when renaming a computed relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10284
diff changeset
  1083
                         ask_confirm=self.verbosity>=2)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1084
        self.cmd_drop_relation_type(oldname, commit=commit)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1085
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1086
    def cmd_add_relation_definition(self, subjtype, rtype, objtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1087
        """register a new relation definition, from its definition found in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1088
        schema definition file
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1089
        """
934
f94e34795586 better variable/attribute names; fix remove_cube, it should not alter the .fs_schema attribute (former .new_schema)
sylvain.thenault@logilab.fr
parents: 676
diff changeset
  1090
        rschema = self.fs_schema.rschema(rtype)
9961
cef58bd36f7b [CWEP002 migration] properly raise exception on (add|drop)_relation_definition for computed relation
Laura Médioni <laura.medioni@logilab.fr>
parents: 9773
diff changeset
  1091
        if rschema.rule:
cef58bd36f7b [CWEP002 migration] properly raise exception on (add|drop)_relation_definition for computed relation
Laura Médioni <laura.medioni@logilab.fr>
parents: 9773
diff changeset
  1092
            raise ExecutionError('Cannot add a relation definition for a '
cef58bd36f7b [CWEP002 migration] properly raise exception on (add|drop)_relation_definition for computed relation
Laura Médioni <laura.medioni@logilab.fr>
parents: 9773
diff changeset
  1093
                                 'computed relation (%s)' % rschema)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1094
        if not rtype in self.repo.schema:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1095
            self.cmd_add_relation_type(rtype, addrdef=False, commit=True)
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1096
        if (subjtype, objtype) in self.repo.schema.rschema(rtype).rdefs:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
  1097
            print('warning: relation %s %s %s is already known, skip addition' % (
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
  1098
                subjtype, rtype, objtype))
7556
867ec36530b8 [migration] closes #1777411: schema modification commands shouldn't crash if already exists
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7505
diff changeset
  1099
            return
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1100
        rdef = self._get_rdef(rschema, subjtype, objtype)
9511
241b1232ed7f Use repoapi instead of dbapi for cwctl shell, upgrade and db-init
Julien Cristau <julien.cristau@logilab.fr>
parents: 9478
diff changeset
  1101
        ss.execschemarql(self.cnx.execute, rdef,
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1102
                         ss.rdef2rql(rdef, self.cstrtype_mapping(),
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1103
                                     self.group_mapping()))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1104
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1105
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1106
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1107
    def _get_rdef(self, rschema, subjtype, objtype):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1108
        return self._set_rdef_eid(rschema.rdefs[(subjtype, objtype)])
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1109
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1110
    def _set_rdef_eid(self, rdef):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1111
        for attr in ('rtype', 'subject', 'object'):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1112
            schemaobj = getattr(rdef, attr)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1113
            if getattr(schemaobj, 'eid', None) is None:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1114
                schemaobj.eid =  self.repo.schema[schemaobj].eid
8278
3e8b94d921e0 [migration] add schema obj into assertion failure message so we get a chance to understand the pb
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8232
diff changeset
  1115
                assert schemaobj.eid is not None, schemaobj
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1116
        return rdef
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
  1117
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1118
    def cmd_drop_relation_definition(self, subjtype, rtype, objtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1119
        """unregister an existing relation definition"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1120
        rschema = self.repo.schema.rschema(rtype)
9961
cef58bd36f7b [CWEP002 migration] properly raise exception on (add|drop)_relation_definition for computed relation
Laura Médioni <laura.medioni@logilab.fr>
parents: 9773
diff changeset
  1121
        if rschema.rule:
cef58bd36f7b [CWEP002 migration] properly raise exception on (add|drop)_relation_definition for computed relation
Laura Médioni <laura.medioni@logilab.fr>
parents: 9773
diff changeset
  1122
            raise ExecutionError('Cannot drop a relation definition for a '
cef58bd36f7b [CWEP002 migration] properly raise exception on (add|drop)_relation_definition for computed relation
Laura Médioni <laura.medioni@logilab.fr>
parents: 9773
diff changeset
  1123
                                 'computed relation (%s)' % rschema)
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1124
        # unregister the definition from CWAttribute or CWRelation
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3582
diff changeset
  1125
        if rschema.final:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1126
            etype = 'CWAttribute'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1127
        else:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1128
            etype = 'CWRelation'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1129
        rql = ('DELETE %s X WHERE X from_entity FE, FE name "%s",'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1130
               'X relation_type RT, RT name "%s", X to_entity TE, TE name "%s"')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1131
        self.rqlexec(rql % (etype, subjtype, rtype, objtype),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1132
                     ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1133
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1134
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1135
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1136
    def cmd_sync_schema_props_perms(self, ertype=None, syncperms=True,
1409
f4dee84a618f sql attributes bugfix
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 1399
diff changeset
  1137
                                    syncprops=True, syncrdefs=True, commit=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1138
        """synchronize the persistent schema against the current definition
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1139
        schema.
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1140
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1141
        `ertype` can be :
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1142
        - None, in that case everything will be synced ;
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1143
        - a string, it should be an entity type or
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1144
          a relation type. In that case, only the corresponding
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1145
          entities / relations will be synced ;
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1146
        - an rdef object to synchronize only this specific relation definition
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1147
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1148
        It will synch common stuff between the definition schema and the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1149
        actual persistent schema, it won't add/remove any entity or relation.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1150
        """
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1151
        assert syncperms or syncprops, 'nothing to do'
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1152
        if ertype is not None:
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1153
            if isinstance(ertype, RelationDefinitionSchema):
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1154
                ertype = ertype.as_triple()
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1155
            if isinstance(ertype, (tuple, list)):
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1156
                assert len(ertype) == 3, 'not a relation definition'
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3836
diff changeset
  1157
                self._synchronize_rdef_schema(ertype[0], ertype[1], ertype[2],
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1158
                                              syncperms=syncperms,
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1159
                                              syncprops=syncprops)
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1160
            else:
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1161
                erschema = self.repo.schema[ertype]
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1162
                if isinstance(erschema, CubicWebRelationSchema):
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1163
                    self._synchronize_rschema(erschema, syncrdefs=syncrdefs,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1164
                                              syncperms=syncperms,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1165
                                              syncprops=syncprops)
4041
be6e473e6b43 fix sync_schema*: new syncprops argument to sync relation definition as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4021
diff changeset
  1166
                else:
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1167
                    self._synchronize_eschema(erschema, syncrdefs=syncrdefs,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1168
                                              syncperms=syncperms,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1169
                                              syncprops=syncprops)
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1170
        else:
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1171
            for etype in self.repo.schema.entities():
7505
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1172
                if etype.eid is None:
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1173
                     # not yet added final etype (thing to BigInt defined in
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1174
                     # yams though 3.13 migration not done yet)
e04790a6eea8 [migration] this is needed to avoid pb with yams 0.33 / cw < 3.13
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
  1175
                    continue
5666
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1176
                self._synchronize_eschema(etype, syncrdefs=syncrdefs,
58d6c2c96500 [migration] fix schema synchronization bug
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5552
diff changeset
  1177
                                          syncprops=syncprops, syncperms=syncperms)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1178
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1179
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1180
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1181
    def cmd_change_relation_props(self, subjtype, rtype, objtype,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1182
                                  commit=True, **kwargs):
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1183
        """change some properties of a relation definition
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1184
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1185
        you usually want to use sync_schema_props_perms instead.
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1186
        """
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1187
        assert kwargs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1188
        restriction = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1189
        if subjtype and subjtype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1190
            restriction.append('X from_entity FE, FE name "%s"' % subjtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1191
        if objtype and objtype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1192
            restriction.append('X to_entity TE, TE name "%s"' % objtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1193
        if rtype and rtype != 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1194
            restriction.append('X relation_type RT, RT name "%s"' % rtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1195
        assert restriction
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1196
        values = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1197
        for k, v in kwargs.items():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1198
            values.append('X %s %%(%s)s' % (k, k))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1199
            if isinstance(v, str):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1200
                kwargs[k] = unicode(v)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1201
        rql = 'SET %s WHERE %s' % (','.join(values), ','.join(restriction))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1202
        self.rqlexec(rql, kwargs, ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1203
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1204
            self.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1205
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1206
    def cmd_set_size_constraint(self, etype, rtype, size, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1207
        """set change size constraint of a string attribute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1208
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1209
        if size is None any size constraint will be removed.
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1210
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1211
        you usually want to use sync_schema_props_perms instead.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1212
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1213
        oldvalue = None
4695
4aaf87e7f79e 3.6 api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4650
diff changeset
  1214
        for constr in self.repo.schema.eschema(etype).rdef(rtype).constraints:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1215
            if isinstance(constr, SizeConstraint):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1216
                oldvalue = constr.max
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1217
        if oldvalue == size:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1218
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1219
        if oldvalue is None and not size is None:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1220
            ceid = self.rqlexec('INSERT CWConstraint C: C value %(v)s, C cstrtype CT '
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1221
                                'WHERE CT name "SizeConstraint"',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1222
                                {'v': SizeConstraint(size).serialize()},
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1223
                                ask_confirm=self.verbosity>=2)[0][0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1224
            self.rqlexec('SET X constrained_by C WHERE X from_entity S, X relation_type R, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1225
                         'S name "%s", R name "%s", C eid %s' % (etype, rtype, ceid),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1226
                         ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1227
        elif not oldvalue is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1228
            if not size is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1229
                self.rqlexec('SET C value %%(v)s WHERE X from_entity S, X relation_type R,'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1230
                             'X constrained_by C, C cstrtype CT, CT name "SizeConstraint",'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1231
                             'S name "%s", R name "%s"' % (etype, rtype),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1232
                             {'v': unicode(SizeConstraint(size).serialize())},
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1233
                             ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1234
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1235
                self.rqlexec('DELETE X constrained_by C WHERE X from_entity S, X relation_type R,'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1236
                             'X constrained_by C, C cstrtype CT, CT name "SizeConstraint",'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1237
                             'S name "%s", R name "%s"' % (etype, rtype),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1238
                             ask_confirm=self.verbosity>=2)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1239
                # cleanup unused constraints
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1240
                self.rqlexec('DELETE CWConstraint C WHERE NOT X constrained_by C')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1241
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1242
            self.commit()
1399
3f408c7a164e unify schema sync migration commands with (hopefuly) a clearer name
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1243
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1244
    # Workflows handling ######################################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1245
6805
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1246
    def cmd_make_workflowable(self, etype):
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1247
        """add workflow relations to an entity type to make it workflowable"""
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1248
        self.cmd_add_relation_definition(etype, 'in_state', 'State')
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1249
        self.cmd_add_relation_definition(etype, 'custom_workflow', 'Workflow')
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1250
        self.cmd_add_relation_definition('TrInfo', 'wf_info_for', etype)
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1251
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1252
    def cmd_add_workflow(self, name, wfof, default=True, commit=False,
6816
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1253
                         ensure_workflowable=True, **kwargs):
3582
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1254
        """
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1255
        create a new workflow and links it to entity types
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1256
         :type name: unicode
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1257
         :param name: name of the workflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1258
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1259
         :type wfof: string or list/tuple of strings
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1260
         :param wfof: entity type(s) having this workflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1261
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1262
         :type default: bool
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1263
         :param default: tells wether this is the default workflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1264
                   for the specified entity type(s); set it to false in
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1265
                   the case of a subworkflow
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1266
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1267
         :rtype: `Workflow`
28547f21308e two bits of docstring
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 3577
diff changeset
  1268
        """
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1269
        wf = self.cmd_create_entity('Workflow', name=unicode(name),
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1270
                                    **kwargs)
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1271
        if not isinstance(wfof, (list, tuple)):
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1272
            wfof = (wfof,)
6805
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1273
        def _missing_wf_rel(etype):
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1274
            return 'missing workflow relations, see make_workflowable(%s)' % etype
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1275
        for etype in wfof:
6805
d0d7345581d4 [migration] add_workflow now ensure the entity type is actually workflowable, and a new make_workflowable migration function is available to make an entity type workflowable (closes #1382720)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6686
diff changeset
  1276
            eschema = self.repo.schema[etype]
10473
23a2fa8cb725 avoid sanitizing warnings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10365
diff changeset
  1277
            etype = unicode(etype)
6816
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1278
            if ensure_workflowable:
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1279
                assert 'in_state' in eschema.subjrels, _missing_wf_rel(etype)
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1280
                assert 'custom_workflow' in eschema.subjrels, _missing_wf_rel(etype)
f61de39cd396 [wf migration] allow to deactivate assertion on wfable entity to quick-fix tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6815
diff changeset
  1281
                assert 'wf_info_for' in eschema.objrels, _missing_wf_rel(etype)
2956
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1282
            rset = self.rqlexec(
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1283
                'SET X workflow_of ET WHERE X eid %(x)s, ET name %(et)s',
10095
200bd6a601dc [warnings] put an end to warnings in the sqlite driver over `str` being sent instead of unicode strings
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10084
diff changeset
  1284
                {'x': wf.eid, 'et': unicode(etype)}, ask_confirm=False)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1285
            assert rset, 'unexistant entity type %s' % etype
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1286
            if default:
2956
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1287
                self.rqlexec(
6a57c0be0e58 [migration] don't ask confirm here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2943
diff changeset
  1288
                    'SET ET default_workflow X WHERE X eid %(x)s, ET name %(et)s',
10095
200bd6a601dc [warnings] put an end to warnings in the sqlite driver over `str` being sent instead of unicode strings
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10084
diff changeset
  1289
                    {'x': wf.eid, 'et': unicode(etype)}, ask_confirm=False)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1290
        if commit:
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1291
            self.commit()
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1292
        return wf
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1293
7295
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1294
    def cmd_get_workflow_for(self, etype):
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1295
        """return default workflow for the given entity type"""
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1296
        rset = self.rqlexec('Workflow X WHERE ET default_workflow X, ET name %(et)s',
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1297
                            {'et': etype})
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1298
        return rset.get_entity(0, 0)
b6fd14ee491e [migration] new command to get default workflow for an entity type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7166
diff changeset
  1299
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1263
diff changeset
  1300
    # CWProperty handling ######################################################
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1301
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1302
    def cmd_property_value(self, pkey):
7111
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1303
        """retreive the site-wide persistent property value for the given key.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1304
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1305
        To get a user specific property value, use appropriate method on CWUser
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1306
        instance.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1307
        """
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1308
        rset = self.rqlexec(
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1309
            'Any V WHERE X is CWProperty, X pkey %(k)s, X value V, NOT X for_user U',
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1310
            {'k': pkey}, ask_confirm=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1311
        return rset[0][0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1312
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1313
    def cmd_set_property(self, pkey, value):
7111
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1314
        """set the site-wide persistent property value for the given key to the
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1315
        given value.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1316
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1317
        To set a user specific property value, use appropriate method on CWUser
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1318
        instance.
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1319
        """
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1320
        value = unicode(value)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1321
        try:
7111
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1322
            prop = self.rqlexec(
6c8e8747268d [cwprops, migration] fix [set_]property to ensure we don't retrieve a user specific value. Add some docstring along the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7008
diff changeset
  1323
                'CWProperty X WHERE X pkey %(k)s, NOT X for_user U',
10095
200bd6a601dc [warnings] put an end to warnings in the sqlite driver over `str` being sent instead of unicode strings
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10084
diff changeset
  1324
                {'k': unicode(pkey)}, ask_confirm=False).get_entity(0, 0)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
  1325
        except Exception:
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1326
            self.cmd_create_entity('CWProperty', pkey=unicode(pkey), value=value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1327
        else:
8483
4ba11607d84a [entity api] unify set_attributes / set_relations into a cw_set method. Closes #2423719
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8467
diff changeset
  1328
            prop.cw_set(value=value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1329
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1330
    # other data migration commands ###########################################
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1331
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1332
    def cmd_storage_changed(self, etype, attribute):
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1333
        """migrate entities to a custom storage. The new storage is expected to
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1334
        be set, it will be temporarily removed for the migration.
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1335
        """
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1336
        from logilab.common.shellutils import ProgressBar
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1337
        source = self.repo.system_source
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1338
        storage = source.storage(etype, attribute)
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1339
        source.unset_storage(etype, attribute)
5552
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5551
diff changeset
  1340
        rset = self.rqlexec('Any X WHERE X is %s' % etype, ask_confirm=False)
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1341
        pb = ProgressBar(len(rset))
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1342
        for entity in rset.entities():
5551
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1343
            # fill cache. Do not fetch that attribute using the global rql query
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1344
            # since we may exhaust memory doing that....
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1345
            getattr(entity, attribute)
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1346
            storage.migrate_entity(entity, attribute)
5551
e41cd3bf5080 [migration] fix storage_change migraction to avoid exhausting the server's memory
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
  1347
            # remove from entity cache to avoid memory exhaustion
8231
1bb43e31032d [bfss, migration] fix storage migration (closes #2189206)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8047
diff changeset
  1348
            del entity.cw_attr_cache[attribute]
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1349
            pb.update()
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
  1350
        print()
5397
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1351
        source.set_storage(etype, attribute, storage)
cdbf823450aa [bfss] new storage_changed migration action to move an attribute to a custom storage. Closes #893941
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5302
diff changeset
  1352
5132
260d73ad4f24 [cleaning] simpler implementation of cmd_create_entity
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5043
diff changeset
  1353
    def cmd_create_entity(self, etype, commit=False, **kwargs):
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1354
        """add a new entity of the given type"""
9511
241b1232ed7f Use repoapi instead of dbapi for cwctl shell, upgrade and db-init
Julien Cristau <julien.cristau@logilab.fr>
parents: 9478
diff changeset
  1355
        entity = self.cnx.create_entity(etype, **kwargs)
2920
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1356
        if commit:
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1357
            self.commit()
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1358
        return entity
64322aa83a1d start a new workflow engine
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2903
diff changeset
  1359
9750
566f8fce5168 [dataimport, migration] silence find_entities / find_one_entity warning
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9686
diff changeset
  1360
    def cmd_find(self, etype, **kwargs):
566f8fce5168 [dataimport, migration] silence find_entities / find_one_entity warning
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9686
diff changeset
  1361
        """find entities of the given type and attribute values"""
566f8fce5168 [dataimport, migration] silence find_entities / find_one_entity warning
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9686
diff changeset
  1362
        return self.cnx.find(etype, **kwargs)
566f8fce5168 [dataimport, migration] silence find_entities / find_one_entity warning
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9686
diff changeset
  1363
566f8fce5168 [dataimport, migration] silence find_entities / find_one_entity warning
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9686
diff changeset
  1364
    @deprecated("[3.19] use find(*args, **kwargs).entities() instead")
7116
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1365
    def cmd_find_entities(self, etype, **kwargs):
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1366
        """find entities of the given type and attribute values"""
9750
566f8fce5168 [dataimport, migration] silence find_entities / find_one_entity warning
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9686
diff changeset
  1367
        return self.cnx.find(etype, **kwargs).entities()
7116
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1368
9750
566f8fce5168 [dataimport, migration] silence find_entities / find_one_entity warning
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9686
diff changeset
  1369
    @deprecated("[3.19] use find(*args, **kwargs).one() instead")
7116
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1370
    def cmd_find_one_entity(self, etype, **kwargs):
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1371
        """find one entity of the given type and attribute values.
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1372
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1373
        raise :exc:`cubicweb.req.FindEntityError` if can not return one and only
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1374
        one entity.
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1375
        """
9750
566f8fce5168 [dataimport, migration] silence find_entities / find_one_entity warning
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9686
diff changeset
  1376
        return self.cnx.find(etype, **kwargs).one()
7116
dfd4680a23f0 [session] add find_entities and find_one_entity to session/request API (closes #1550045)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7111
diff changeset
  1377
5928
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1378
    def cmd_update_etype_fti_weight(self, etype, weight):
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1379
        if self.repo.system_source.dbdriver == 'postgres':
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1380
            self.sqlexec('UPDATE appears SET weight=%(weight)s '
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1381
                         'FROM entities as X '
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1382
                         'WHERE X.eid=appears.uid AND X.type=%(type)s',
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1383
                         {'type': etype, 'weight': weight}, ask_confirm=False)
d9936c39d478 [migration] new migration command to change fti weight for an entity t ype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5897
diff changeset
  1384
5850
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1385
    def cmd_reindex_entities(self, etypes=None):
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1386
        """force reindexaction of entities of the given types or of all
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1387
        indexable entity types
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1388
        """
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1389
        from cubicweb.server.checkintegrity import reindex_entities
10353
d9a1e7939ee6 [migractions] remove any session related leftovers
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10346
diff changeset
  1390
        reindex_entities(self.repo.schema, self.cnx, etypes=etypes)
5850
fabff2813ee4 [migration] schema should be accessed through .repo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5666
diff changeset
  1391
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1392
    @contextmanager
7409
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1393
    def cmd_dropped_constraints(self, etype, attrname, cstrtype=None,
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1394
                                droprequired=False):
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1395
        """context manager to drop constraints temporarily on fs_schema
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1396
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1397
        `cstrtype` should be a constraint class (or a tuple of classes)
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1398
        and will be passed to isinstance directly
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1399
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1400
        For instance::
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1401
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1402
            >>> with dropped_constraints('MyType', 'myattr',
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1403
            ...                          UniqueConstraint, droprequired=True):
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1404
            ...     add_attribute('MyType', 'myattr')
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1405
            ...     # + instructions to fill MyType.myattr column
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1406
            ...
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1407
            >>>
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1408
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1409
        """
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1410
        rdef = self.fs_schema.eschema(etype).rdef(attrname)
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1411
        original_constraints = rdef.constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1412
        # remove constraints
7409
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1413
        if cstrtype:
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1414
            rdef.constraints = [cstr for cstr in original_constraints
47c8009dbbe2 [migractions] dropped_constraints should accept only droprequired without cstrtype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7342
diff changeset
  1415
                                if not (cstrtype and isinstance(cstr, cstrtype))]
6292
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1416
        if droprequired:
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1417
            original_cardinality = rdef.cardinality
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1418
            rdef.cardinality = '?' + rdef.cardinality[1]
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1419
        yield
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1420
        # restore original constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1421
        rdef.constraints = original_constraints
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1422
        if droprequired:
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1423
            rdef.cardinality = original_cardinality
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1424
        # update repository schema
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1425
        self.cmd_sync_schema_props_perms(rdef, syncperms=False)
054fa36060d5 [migractions] add a dropped_constraints() migration command
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 6217
diff changeset
  1426
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1427
    def sqlexec(self, sql, args=None, ask_confirm=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1428
        """execute the given sql if confirmed
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1429
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1430
        should only be used for low level stuff undoable with existing higher
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1431
        level actions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1432
        """
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1433
        if not ask_confirm or self.confirm('Execute sql: %s ?' % sql):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1434
            try:
9555
370a7c40864f [server] use the ClientConnection directly now that it has more methods available
Julien Cristau <julien.cristau@logilab.fr>
parents: 9511
diff changeset
  1435
                cu = self.cnx.system_sql(sql, args)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
  1436
            except Exception:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1437
                ex = sys.exc_info()[1]
6187
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6186
diff changeset
  1438
                if self.confirm('Error: %s\nabort?' % ex, pdb=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1439
                    raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1440
                return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1441
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1442
                return cu.fetchall()
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
  1443
            except Exception:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1444
                # no result to fetch
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1445
                return
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1446
9255
46f41c3e1443 remove 3.8 bw compat
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9241
diff changeset
  1447
    def rqlexec(self, rql, kwargs=None, build_descr=True,
6186
8ada7df95877 [migration] set ask_confirm=False by default on rqlexec
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6179
diff changeset
  1448
                ask_confirm=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1449
        """rql action"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1450
        if not isinstance(rql, (tuple, list)):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1451
            rql = ( (rql, kwargs), )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1452
        res = None
9511
241b1232ed7f Use repoapi instead of dbapi for cwctl shell, upgrade and db-init
Julien Cristau <julien.cristau@logilab.fr>
parents: 9478
diff changeset
  1453
        execute = self.cnx.execute
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1454
        for rql, kwargs in rql:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1455
            if kwargs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1456
                msg = '%s (%s)' % (rql, kwargs)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1457
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1458
                msg = rql
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1459
            if not ask_confirm or self.confirm('Execute rql: %s ?' % msg):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1460
                try:
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5132
diff changeset
  1461
                    res = execute(rql, kwargs, build_descr=build_descr)
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8694
diff changeset
  1462
                except Exception as ex:
6187
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6186
diff changeset
  1463
                    if self.confirm('Error: %s\nabort?' % ex, pdb=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1464
                        raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1465
        return res
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1466
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1467
    def rqliter(self, rql, kwargs=None, ask_confirm=True):
6686
070e3b23160d [migration] fix rqliter when some query arguments are given
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6582
diff changeset
  1468
        return ForRqlIterator(self, rql, kwargs, ask_confirm)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1469
8181
166ad5458ed8 [migractions] fix comment to avoid ambiguity. Closes #2015260
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8049
diff changeset
  1470
    # low-level commands to repair broken system database ######################
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1471
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1472
    def cmd_change_attribute_type(self, etype, attr, newtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1473
        """low level method to change the type of an entity attribute. This is
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1474
        a quick hack which has some drawback:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1475
        * only works when the old type can be changed to the new type by the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1476
          underlying rdbms (eg using ALTER TABLE)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1477
        * the actual schema won't be updated until next startup
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1478
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1479
        rschema = self.repo.schema.rschema(attr)
10645
57c60a96de70 [migration] fix change_attribute_type to update the live schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10573
diff changeset
  1480
        oldschema = rschema.objects(etype)[0]
57c60a96de70 [migration] fix change_attribute_type to update the live schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10573
diff changeset
  1481
        rdef = rschema.rdef(etype, oldschema)
7410
1dbf9a83bb49 [migractions] fix change_attribute_type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7409
diff changeset
  1482
        sql = ("UPDATE cw_CWAttribute "
1dbf9a83bb49 [migractions] fix change_attribute_type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7409
diff changeset
  1483
               "SET cw_to_entity=(SELECT cw_eid FROM cw_CWEType WHERE cw_name='%s')"
10645
57c60a96de70 [migration] fix change_attribute_type to update the live schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10573
diff changeset
  1484
               "WHERE cw_eid=%s") % (newtype, rdef.eid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1485
        self.sqlexec(sql, ask_confirm=False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1486
        dbhelper = self.repo.system_source.dbhelper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1487
        sqltype = dbhelper.TYPE_MAPPING[newtype]
10353
d9a1e7939ee6 [migractions] remove any session related leftovers
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10346
diff changeset
  1488
        cursor = self.cnx.cnxset.cu
10645
57c60a96de70 [migration] fix change_attribute_type to update the live schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10573
diff changeset
  1489
        allownull = rdef.cardinality[0] != '1'
57c60a96de70 [migration] fix change_attribute_type to update the live schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10573
diff changeset
  1490
        dbhelper.change_col_type(cursor, 'cw_%s' % etype, 'cw_%s' % attr, sqltype, allownull)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1491
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1492
            self.commit()
10645
57c60a96de70 [migration] fix change_attribute_type to update the live schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10573
diff changeset
  1493
            # manually update live schema
57c60a96de70 [migration] fix change_attribute_type to update the live schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10573
diff changeset
  1494
            eschema = self.repo.schema[etype]
57c60a96de70 [migration] fix change_attribute_type to update the live schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10573
diff changeset
  1495
            rschema._subj_schemas[eschema].remove(oldschema)
57c60a96de70 [migration] fix change_attribute_type to update the live schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10573
diff changeset
  1496
            rschema._obj_schemas[oldschema].remove(eschema)
57c60a96de70 [migration] fix change_attribute_type to update the live schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10573
diff changeset
  1497
            newschema = self.repo.schema[newtype]
57c60a96de70 [migration] fix change_attribute_type to update the live schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10573
diff changeset
  1498
            rschema._update(eschema, newschema)
57c60a96de70 [migration] fix change_attribute_type to update the live schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10573
diff changeset
  1499
            rdef.object = newschema
57c60a96de70 [migration] fix change_attribute_type to update the live schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10573
diff changeset
  1500
            del rschema.rdefs[(eschema, oldschema)]
57c60a96de70 [migration] fix change_attribute_type to update the live schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10573
diff changeset
  1501
            rschema.rdefs[(eschema, newschema)] = rdef
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1502
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1503
    def cmd_add_entity_type_table(self, etype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1504
        """low level method to create the sql table for an existing entity.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1505
        This may be useful on accidental desync between the repository schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1506
        and a sql database
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1507
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1508
        dbhelper = self.repo.system_source.dbhelper
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
  1509
        tablesql = eschema2sql(dbhelper, self.repo.schema.eschema(etype),
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1240
diff changeset
  1510
                               prefix=SQL_PREFIX)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1511
        for sql in tablesql.split(';'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1512
            if sql.strip():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1513
                self.sqlexec(sql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1514
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1515
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1516
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1517
    def cmd_add_relation_type_table(self, rtype, commit=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1518
        """low level method to create the sql table for an existing relation.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1519
        This may be useful on accidental desync between the repository schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1520
        and a sql database
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1521
        """
7879
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
  1522
        tablesql = rschema2sql(self.repo.schema.rschema(rtype))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1523
        for sql in tablesql.split(';'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1524
            if sql.strip():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1525
                self.sqlexec(sql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1526
        if commit:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1527
            self.commit()
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1528
8530
2bceea9dee95 [migration] fix backward compat for rename_relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8467
diff changeset
  1529
    @deprecated("[3.15] use rename_relation_type(oldname, newname)")
8467
ad75430a2dc8 [migraction] rename rename_relation method to rename_relation_type for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8307
diff changeset
  1530
    def cmd_rename_relation(self, oldname, newname, commit=True):
8530
2bceea9dee95 [migration] fix backward compat for rename_relation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8467
diff changeset
  1531
        self.cmd_rename_relation_type(oldname, newname, commit)
8467
ad75430a2dc8 [migraction] rename rename_relation method to rename_relation_type for consistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8307
diff changeset
  1532
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1533
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1534
class ForRqlIterator:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1535
    """specific rql iterator to make the loop skipable"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1536
    def __init__(self, helper, rql, kwargs, ask_confirm):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1537
        self._h = helper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1538
        self.rql = rql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1539
        self.kwargs = kwargs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1540
        self.ask_confirm = ask_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1541
        self._rsetit = None
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1542
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1543
    def __iter__(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1544
        return self
1477
b056a49c16dc backport default branch
sylvain.thenault@logilab.fr
parents: 1414 1278
diff changeset
  1545
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1546
    def _get_rset(self):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1547
        rql, kwargs = self.rql, self.kwargs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1548
        if kwargs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1549
            msg = '%s (%s)' % (rql, kwargs)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1550
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1551
            msg = rql
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1552
        if self.ask_confirm:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1553
            if not self._h.confirm('Execute rql: %s ?' % msg):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1554
                raise StopIteration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1555
        try:
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1556
            return self._h._cw.execute(rql, kwargs)
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8694
diff changeset
  1557
        except Exception as ex:
4650
965395d821bc typo: capitalize migration confirmation messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4640
diff changeset
  1558
            if self._h.confirm('Error: %s\nabort?' % ex):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1559
                raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1560
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1561
                raise StopIteration
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1562
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1563
    def next(self):
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1564
        if self._rsetit is not None:
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1565
            return self._rsetit.next()
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1566
        rset = self._get_rset()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1567
        self._rsetit = iter(rset)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1568
        return self._rsetit.next()
6125
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1569
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1570
    def entities(self):
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1571
        try:
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1572
            rset = self._get_rset()
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1573
        except StopIteration:
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1574
            return []
46b9cbbc1e37 [migration] enhance RqlForIterator so we can do 'for e in rqliter(...).entities()'
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6118
diff changeset
  1575
        return rset.entities()