cubicweb/migration.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Thu, 30 Jun 2016 13:26:36 +0200
changeset 11408 5be298ed4caa
parent 11286 0b38e373b985
child 11767 432f87a63057
permissions -rw-r--r--
[schema2sql] More docstrings and minor API improvements Along the way, renamed local `column` variable to `attr` where we're NOT manipulating a DB column name but a schema attribute. Related to #14050899
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7576
diff changeset
     1
# copyright 2003-2011 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
5967
8deedfeb7846 cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5590
diff changeset
    18
"""utilities for instances migration"""
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
    19
from __future__ import print_function
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
import sys
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
import os
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
import logging
2446
440cb4ea7e5c [refactor] #342855: replace uses of (deprecated) mktemp
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2275
diff changeset
    26
import tempfile
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
from os.path import exists, join, basename, splitext
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
    28
from itertools import chain
10802
3d948d35d94f [migration] re-try without print_function when compiling a script fails
Julien Cristau <julien.cristau@logilab.fr>
parents: 10720
diff changeset
    29
from warnings import warn
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    30
10612
84468b90e9c1 [py3k] basestring → six.string_types
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    31
from six import string_types
84468b90e9c1 [py3k] basestring → six.string_types
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    32
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
    33
from logilab.common import IGNORED_EXTENSIONS
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    34
from logilab.common.decorators import cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    35
from logilab.common.configuration import REQUIRED, read_old_config
2615
1ea41b7c0836 F [dialog] offer to create backup. refactor to use l.c.shellutils.ASK
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2512
diff changeset
    36
from logilab.common.shellutils import ASK
5027
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
    37
from logilab.common.changelog import Version
10107
1ef92a6193a8 [serverctl] rename remove_cube to drop_cube (closes #4545093)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 10074
diff changeset
    38
from logilab.common.deprecation import deprecated
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    39
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
    40
from cubicweb import ConfigurationError, ExecutionError
6657
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
    41
from cubicweb.cwconfig import CubicWebConfiguration as cwcfg
8056
8909800a8c51 [cleanup] drop some appengine support junk
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7879
diff changeset
    42
from cubicweb.toolsutils import show_diffs
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    43
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    44
def filter_scripts(config, directory, fromversion, toversion, quiet=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    45
    """return a list of paths of migration files to consider to upgrade
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    46
    from a version to a greater one
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    47
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    48
    from logilab.common.changelog import Version # doesn't work with appengine
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    49
    assert fromversion
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
    assert toversion
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    51
    assert isinstance(fromversion, tuple), fromversion.__class__
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
    assert isinstance(toversion, tuple), toversion.__class__
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    53
    assert fromversion <= toversion, (fromversion, toversion)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    54
    if not exists(directory):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    55
        if not quiet:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
    56
            print(directory, "doesn't exists, no migration path")
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    57
        return []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    58
    if fromversion == toversion:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    59
        return []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    60
    result = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    61
    for fname in os.listdir(directory):
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
    62
        if fname.endswith(IGNORED_EXTENSIONS):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
            continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    64
        fpath = join(directory, fname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    65
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    66
            tver, mode = fname.split('_', 1)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    67
        except ValueError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    68
            continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    69
        mode = mode.split('.', 1)[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    70
        if not config.accept_mode(mode):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    71
            continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    72
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    73
            tver = Version(tver)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    74
        except ValueError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    75
            continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    76
        if tver <= fromversion:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    77
            continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    78
        if tver > toversion:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    79
            continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    80
        result.append((tver, fpath))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    81
    # be sure scripts are executed in order
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    82
    return sorted(result)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    83
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    84
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    85
def execscript_confirm(scriptpath):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
    """asks for confirmation before executing a script and provides the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
    ability to show the script's content
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    88
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    89
    while True:
4553
23201259ffeb [migration] abort becomes possible when asked for confirmation before migration script
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4252
diff changeset
    90
        answer = ASK.ask('Execute %r ?' % scriptpath,
23201259ffeb [migration] abort becomes possible when asked for confirmation before migration script
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4252
diff changeset
    91
                         ('Y','n','show','abort'), 'Y')
23201259ffeb [migration] abort becomes possible when asked for confirmation before migration script
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4252
diff changeset
    92
        if answer == 'abort':
23201259ffeb [migration] abort becomes possible when asked for confirmation before migration script
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4252
diff changeset
    93
            raise SystemExit(1)
23201259ffeb [migration] abort becomes possible when asked for confirmation before migration script
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 4252
diff changeset
    94
        elif answer == 'n':
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
            return False
2615
1ea41b7c0836 F [dialog] offer to create backup. refactor to use l.c.shellutils.ASK
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2512
diff changeset
    96
        elif answer == 'show':
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    97
            stream = open(scriptpath)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    98
            scriptcontent = stream.read()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    99
            stream.close()
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   100
            print()
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   101
            print(scriptcontent)
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   102
            print()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   103
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   104
            return True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   105
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   106
def yes(*args, **kwargs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   107
    return True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   108
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   109
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   110
class MigrationHelper(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   111
    """class holding CubicWeb Migration Actions used by migration scripts"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   112
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   113
    def __init__(self, config, interactive=True, verbosity=1):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   114
        self.config = config
3700
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3230
diff changeset
   115
        if config:
fd550e4dc515 #481017: cubicweb-ctl shell on remote instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3230
diff changeset
   116
            # no config on shell to a remote instance
5442
3ed8afbbdf70 [webconfig] refactor/cleanup debug mode management on startup: simply use config.debugmode instead of debug argument everywhere...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5436
diff changeset
   117
            self.config.init_log(logthreshold=logging.ERROR)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   118
        # 0: no confirmation, 1: only main commands confirmed, 2 ask for everything
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   119
        self.verbosity = verbosity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   120
        self.need_wrap = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   121
        if not interactive or not verbosity:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   122
            self.confirm = yes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   123
            self.execscript_confirm = yes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   124
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   125
            self.execscript_confirm = execscript_confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   126
        self._option_changes = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   127
        self.__context = {'confirm': self.confirm,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   128
                          'config': self.config,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
                          'interactive_mode': interactive,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   130
                          }
6217
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6187
diff changeset
   131
        self._context_stack = []
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   132
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   133
    def __getattribute__(self, name):
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   134
        try:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   135
            return object.__getattribute__(self, name)
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   136
        except AttributeError:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   137
            cmd = 'cmd_%s' % name
6362
1b5fc8581437 [c-c] fix RuntimeError: 'maximum recursion depth exceeded while calling a Python object' we get when creating/upgrading/shelling an instance: hasattr() call __getattribute__, creating an infinite recursion error catched by the interpretor. Avoid this by testing the method is available on the class instead of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6221
diff changeset
   138
            # search self.__class__ to avoid infinite recursion
1b5fc8581437 [c-c] fix RuntimeError: 'maximum recursion depth exceeded while calling a Python object' we get when creating/upgrading/shelling an instance: hasattr() call __getattribute__, creating an infinite recursion error catched by the interpretor. Avoid this by testing the method is available on the class instead of the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6221
diff changeset
   139
            if hasattr(self.__class__, cmd):
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   140
                meth = getattr(self, cmd)
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   141
                return lambda *args, **kwargs: self.interact(args, kwargs,
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   142
                                                             meth=meth)
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   143
            raise
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   144
        raise AttributeError(name)
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   145
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   146
    def migrate(self, vcconf, toupgrade, options):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   147
        """upgrade the given set of cubes
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 676
diff changeset
   148
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   149
        `cubes` is an ordered list of 3-uple:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   150
        (cube, fromversion, toversion)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   151
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   152
        if options.fs_only:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   153
            # monkey path configuration.accept_mode so database mode (e.g. Any)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   154
            # won't be accepted
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   155
            orig_accept_mode = self.config.accept_mode
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   156
            def accept_mode(mode):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   157
                if mode == 'Any':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   158
                    return False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   159
                return orig_accept_mode(mode)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
            self.config.accept_mode = accept_mode
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   161
        # may be an iterator
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   162
        toupgrade = tuple(toupgrade)
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   163
        vmap = dict( (cube, (fromver, tover)) for cube, fromver, tover in toupgrade)
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   164
        ctx = self.__context
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   165
        ctx['versions_map'] = vmap
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   166
        if self.config.accept_mode('Any') and 'cubicweb' in vmap:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   167
            migrdir = self.config.migration_scripts_dir()
3715
e3ccadb126d7 [shell] make process_script available throuhg c-c shell / migration script context
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3700
diff changeset
   168
            self.cmd_process_script(join(migrdir, 'bootstrapmigration_repository.py'))
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   169
        for cube, fromversion, toversion in toupgrade:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   170
            if cube == 'cubicweb':
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   171
                migrdir = self.config.migration_scripts_dir()
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   172
            else:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   173
                migrdir = self.config.cube_migration_scripts_dir(cube)
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   174
            scripts = filter_scripts(self.config, migrdir, fromversion, toversion)
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   175
            if scripts:
2481
24bad65dbebd take care to migration w/ X.Y.Z_Any.py / X.Y.Z_common.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   176
                prevversion = None
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   177
                for version, script in scripts:
2481
24bad65dbebd take care to migration w/ X.Y.Z_Any.py / X.Y.Z_common.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   178
                    # take care to X.Y.Z_Any.py / X.Y.Z_common.py: we've to call
24bad65dbebd take care to migration w/ X.Y.Z_Any.py / X.Y.Z_common.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   179
                    # cube_upgraded once all script of X.Y.Z have been executed
24bad65dbebd take care to migration w/ X.Y.Z_Any.py / X.Y.Z_common.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   180
                    if prevversion is not None and version != prevversion:
2897
2658f432284c [migration] consider previous version as done, not current version
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2738
diff changeset
   181
                        self.cube_upgraded(cube, prevversion)
2481
24bad65dbebd take care to migration w/ X.Y.Z_Any.py / X.Y.Z_common.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   182
                    prevversion = version
3715
e3ccadb126d7 [shell] make process_script available throuhg c-c shell / migration script context
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3700
diff changeset
   183
                    self.cmd_process_script(script)
2481
24bad65dbebd take care to migration w/ X.Y.Z_Any.py / X.Y.Z_common.py
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   184
                self.cube_upgraded(cube, toversion)
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   185
            else:
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   186
                self.cube_upgraded(cube, toversion)
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   187
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   188
    def cube_upgraded(self, cube, version):
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2124
diff changeset
   189
        pass
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   190
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   191
    def shutdown(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   192
        pass
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 676
diff changeset
   193
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   194
    def interact(self, args, kwargs, meth):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   195
        """execute the given method according to user's confirmation"""
2512
106b2a05dc88 [cleanup] started to improve cubicweb-ctl dialog messages consistency
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2481
diff changeset
   196
        msg = 'Execute command: %s(%s) ?' % (
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   197
            meth.__name__[4:],
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   198
            ', '.join([repr(arg) for arg in args] +
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   199
                      ['%s=%r' % (n,v) for n,v in kwargs.items()]))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   200
        if 'ask_confirm' in kwargs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   201
            ask_confirm = kwargs.pop('ask_confirm')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   202
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   203
            ask_confirm = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   204
        if not ask_confirm or self.confirm(msg):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   205
            return meth(*args, **kwargs)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   206
7879
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   207
    def confirm(self, question, # pylint: disable=E0202
9aae456abab5 [pylint] fix pylint detected errors and tweak it so that pylint -E will be much less verbose next time (+ update some copyrights on the way)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7815
diff changeset
   208
                shell=True, abort=True, retry=False, pdb=False, default='y'):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   209
        """ask for confirmation and return true on positive answer
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   210
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   211
        if `retry` is true the r[etry] answer may return 2
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   212
        """
4721
8f63691ccb7f pylint style fixes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4553
diff changeset
   213
        possibleanswers = ['y', 'n']
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   214
        if abort:
2615
1ea41b7c0836 F [dialog] offer to create backup. refactor to use l.c.shellutils.ASK
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2512
diff changeset
   215
            possibleanswers.append('abort')
6187
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
   216
        if pdb:
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
   217
            possibleanswers.append('pdb')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   218
        if shell:
2615
1ea41b7c0836 F [dialog] offer to create backup. refactor to use l.c.shellutils.ASK
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2512
diff changeset
   219
            possibleanswers.append('shell')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   220
        if retry:
2615
1ea41b7c0836 F [dialog] offer to create backup. refactor to use l.c.shellutils.ASK
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2512
diff changeset
   221
            possibleanswers.append('retry')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   222
        try:
2738
e7e46121a4f9 [migration] mh.confirm support a new default argument, fix default handling for ASK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2615
diff changeset
   223
            answer = ASK.ask(question, possibleanswers, default)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   224
        except (EOFError, KeyboardInterrupt):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   225
            answer = 'abort'
2615
1ea41b7c0836 F [dialog] offer to create backup. refactor to use l.c.shellutils.ASK
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2512
diff changeset
   226
        if answer == 'n':
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   227
            return False
2615
1ea41b7c0836 F [dialog] offer to create backup. refactor to use l.c.shellutils.ASK
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2512
diff changeset
   228
        if answer == 'retry':
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   229
            return 2
2615
1ea41b7c0836 F [dialog] offer to create backup. refactor to use l.c.shellutils.ASK
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2512
diff changeset
   230
        if answer == 'abort':
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   231
            raise SystemExit(1)
6187
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
   232
        if answer == 'shell':
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   233
            self.interactive_shell()
6187
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
   234
            return self.confirm(question, shell, abort, retry, pdb, default)
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
   235
        if answer == 'pdb':
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
   236
            import pdb
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
   237
            pdb.set_trace()
348c7d93cda3 [migration] propose to open pdb on error during execution of rql/sql query
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6042
diff changeset
   238
            return self.confirm(question, shell, abort, retry, pdb, default)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   239
        return True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   240
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   241
    def interactive_shell(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   242
        self.confirm = yes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   243
        self.need_wrap = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   244
        # avoid '_' to be added to builtins by sys.display_hook
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   245
        def do_not_add___to_builtins(obj):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   246
            if obj is not None:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   247
                print(repr(obj))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   248
        sys.displayhook = do_not_add___to_builtins
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   249
        local_ctx = self._create_context()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   250
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   251
            import readline
9740
c0239d8ae742 [shell] provide autocompletion for rql queries in cubicweb-ctl shell (closes #3560541)
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 9294
diff changeset
   252
            from cubicweb.toolsutils import CWShellCompleter
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   253
        except ImportError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   254
            # readline not available
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   255
            pass
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 676
diff changeset
   256
        else:
9740
c0239d8ae742 [shell] provide autocompletion for rql queries in cubicweb-ctl shell (closes #3560541)
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 9294
diff changeset
   257
            rql_completer = CWShellCompleter(local_ctx)
c0239d8ae742 [shell] provide autocompletion for rql queries in cubicweb-ctl shell (closes #3560541)
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 9294
diff changeset
   258
            readline.set_completer(rql_completer.complete)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   259
            readline.parse_and_bind('tab: complete')
3201
8af05e82510c HOME is spelled USERPROFILE on windows
Aurelien Campeas
parents: 2897
diff changeset
   260
            home_key = 'HOME'
8af05e82510c HOME is spelled USERPROFILE on windows
Aurelien Campeas
parents: 2897
diff changeset
   261
            if sys.platform == 'win32':
8af05e82510c HOME is spelled USERPROFILE on windows
Aurelien Campeas
parents: 2897
diff changeset
   262
                home_key = 'USERPROFILE'
9294
769b8867f8ce [cwshell] rename history file to .cwshell_history (erudi is dead, long live cw)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8532
diff changeset
   263
            histfile = os.path.join(os.environ[home_key], ".cwshell_history")
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   264
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   265
                readline.read_history_file(histfile)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   266
            except IOError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   267
                pass
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   268
        from code import interact
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   269
        banner = """entering the migration python shell
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   270
just type migration commands or arbitrary python code and type ENTER to execute it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   271
type "exit" or Ctrl-D to quit the shell and resume operation"""
11286
0b38e373b985 [migration] Drop custom "readfunc" for code.interact in shell
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11057
diff changeset
   272
        interact(banner, local=local_ctx)
8108
91868ec92391 [migration] Ignore IOError from readline.write_history_file (closes #2106621)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8107
diff changeset
   273
        try:
91868ec92391 [migration] Ignore IOError from readline.write_history_file (closes #2106621)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8107
diff changeset
   274
            readline.write_history_file(histfile)
91868ec92391 [migration] Ignore IOError from readline.write_history_file (closes #2106621)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8107
diff changeset
   275
        except IOError:
91868ec92391 [migration] Ignore IOError from readline.write_history_file (closes #2106621)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8107
diff changeset
   276
            pass
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   277
        # delete instance's confirm attribute to avoid questions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   278
        del self.confirm
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   279
        self.need_wrap = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   280
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   281
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   282
    def _create_context(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   283
        """return a dictionary to use as migration script execution context"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   284
        context = self.__context
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   285
        for attr in dir(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   286
            if attr.startswith('cmd_'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   287
                if self.need_wrap:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   288
                    context[attr[4:]] = getattr(self, attr[4:])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   289
                else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
                    context[attr[4:]] = getattr(self, attr)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   291
        return context
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 676
diff changeset
   292
6217
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6187
diff changeset
   293
    def update_context(self, key, value):
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6187
diff changeset
   294
        for context in self._context_stack:
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6187
diff changeset
   295
            context[key] = value
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6187
diff changeset
   296
        self.__context[key] = value
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6187
diff changeset
   297
3715
e3ccadb126d7 [shell] make process_script available throuhg c-c shell / migration script context
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3700
diff changeset
   298
    def cmd_process_script(self, migrscript, funcname=None, *args, **kwargs):
5430
ed8f71e244f8 [shell] #715938: support of script parameters (using standard '--' as arguments separator)
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5426
diff changeset
   299
        """execute a migration script in interactive mode
ed8f71e244f8 [shell] #715938: support of script parameters (using standard '--' as arguments separator)
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5426
diff changeset
   300
ed8f71e244f8 [shell] #715938: support of script parameters (using standard '--' as arguments separator)
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5426
diff changeset
   301
        Display the migration script path, ask for confirmation and execute it
ed8f71e244f8 [shell] #715938: support of script parameters (using standard '--' as arguments separator)
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5426
diff changeset
   302
        if confirmed
ed8f71e244f8 [shell] #715938: support of script parameters (using standard '--' as arguments separator)
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5426
diff changeset
   303
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   304
        Allowed input file formats for migration scripts:
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   305
        - `python` (.py)
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   306
        - `sql` (.sql)
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   307
        - `doctest` (.txt or .rst)
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   308
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   309
        .. warning:: sql migration scripts are not available in web-only instance
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   310
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   311
        You can pass script parameters with using double dash (--) in the
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   312
        command line
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   313
5430
ed8f71e244f8 [shell] #715938: support of script parameters (using standard '--' as arguments separator)
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5426
diff changeset
   314
        Context environment can have these variables defined:
5436
2455ca3a2a3a [c-c shell] make script arguments available as __args__ in the script namespace. Use scriptargs instead of args as process_script argument name.
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5430
diff changeset
   315
        - __name__ : will be determine by funcname parameter
2455ca3a2a3a [c-c shell] make script arguments available as __args__ in the script namespace. Use scriptargs instead of args as process_script argument name.
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5430
diff changeset
   316
        - __file__ : is the name of the script if it exists
2455ca3a2a3a [c-c shell] make script arguments available as __args__ in the script namespace. Use scriptargs instead of args as process_script argument name.
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5430
diff changeset
   317
        - __args__ : script arguments coming from command-line
5430
ed8f71e244f8 [shell] #715938: support of script parameters (using standard '--' as arguments separator)
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5426
diff changeset
   318
ed8f71e244f8 [shell] #715938: support of script parameters (using standard '--' as arguments separator)
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5426
diff changeset
   319
        :param migrscript: name of the script
5436
2455ca3a2a3a [c-c shell] make script arguments available as __args__ in the script namespace. Use scriptargs instead of args as process_script argument name.
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5430
diff changeset
   320
        :param funcname: defines __name__ inside the shell (or use __main__)
2455ca3a2a3a [c-c shell] make script arguments available as __args__ in the script namespace. Use scriptargs instead of args as process_script argument name.
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5430
diff changeset
   321
        :params args: optional arguments for funcname
2455ca3a2a3a [c-c shell] make script arguments available as __args__ in the script namespace. Use scriptargs instead of args as process_script argument name.
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5430
diff changeset
   322
        :keyword scriptargs: optional arguments of the script
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
        """
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   324
        ftypes = {'python':  ('.py',),
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   325
                  'doctest': ('.txt', '.rst'),
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   326
                  'sql':     ('.sql',)}
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   327
        # sql migration scripts are not available in web-only instance
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   328
        if not hasattr(self, "session"):
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   329
            ftypes.pop('sql')
3812
d37d7105e15f [B] migration: normalize migration script path
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 3715
diff changeset
   330
        migrscript = os.path.normpath(migrscript)
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   331
        for (script_mode, ftype) in ftypes.items():
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   332
            if migrscript.endswith(ftype):
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   333
                break
3935
2fbb79054a1a imported patch cwctl-shell-textfile
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3812
diff changeset
   334
        else:
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   335
            ftypes = ', '.join(chain(*ftypes.values()))
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   336
            msg = 'ignoring %s, not a valid script extension (%s)'
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   337
            raise ExecutionError(msg % (migrscript, ftypes))
3935
2fbb79054a1a imported patch cwctl-shell-textfile
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3812
diff changeset
   338
        if not self.execscript_confirm(migrscript):
2fbb79054a1a imported patch cwctl-shell-textfile
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3812
diff changeset
   339
            return
2fbb79054a1a imported patch cwctl-shell-textfile
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3812
diff changeset
   340
        scriptlocals = self._create_context().copy()
6696
160ca95eb4cc [migration] put some migration context info in doctest environment: __file__ and __args__
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6687
diff changeset
   341
        scriptlocals.update({'__file__': migrscript,
160ca95eb4cc [migration] put some migration context info in doctest environment: __file__ and __args__
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6687
diff changeset
   342
                             '__args__': kwargs.pop("scriptargs", [])})
6217
e2aeb40d5983 [migration] fix so that context modification are reflected on the currently executed migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6187
diff changeset
   343
        self._context_stack.append(scriptlocals)
3935
2fbb79054a1a imported patch cwctl-shell-textfile
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3812
diff changeset
   344
        if script_mode == 'python':
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
            if funcname is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   346
                pyname = '__main__'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   347
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   348
                pyname = splitext(basename(migrscript))[0]
6696
160ca95eb4cc [migration] put some migration context info in doctest environment: __file__ and __args__
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6687
diff changeset
   349
            scriptlocals['__name__'] = pyname
10615
6c497fe389d2 [py3k] execfile → exec
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10612
diff changeset
   350
            with open(migrscript, 'rb') as fobj:
10802
3d948d35d94f [migration] re-try without print_function when compiling a script fails
Julien Cristau <julien.cristau@logilab.fr>
parents: 10720
diff changeset
   351
                fcontent = fobj.read()
3d948d35d94f [migration] re-try without print_function when compiling a script fails
Julien Cristau <julien.cristau@logilab.fr>
parents: 10720
diff changeset
   352
            try:
3d948d35d94f [migration] re-try without print_function when compiling a script fails
Julien Cristau <julien.cristau@logilab.fr>
parents: 10720
diff changeset
   353
                code = compile(fcontent, migrscript, 'exec')
3d948d35d94f [migration] re-try without print_function when compiling a script fails
Julien Cristau <julien.cristau@logilab.fr>
parents: 10720
diff changeset
   354
            except SyntaxError:
3d948d35d94f [migration] re-try without print_function when compiling a script fails
Julien Cristau <julien.cristau@logilab.fr>
parents: 10720
diff changeset
   355
                # try without print_function
3d948d35d94f [migration] re-try without print_function when compiling a script fails
Julien Cristau <julien.cristau@logilab.fr>
parents: 10720
diff changeset
   356
                code = compile(fcontent, migrscript, 'exec', 0, True)
3d948d35d94f [migration] re-try without print_function when compiling a script fails
Julien Cristau <julien.cristau@logilab.fr>
parents: 10720
diff changeset
   357
                warn('[3.22] script %r should be updated to work with print_function'
3d948d35d94f [migration] re-try without print_function when compiling a script fails
Julien Cristau <julien.cristau@logilab.fr>
parents: 10720
diff changeset
   358
                     % migrscript, DeprecationWarning)
10615
6c497fe389d2 [py3k] execfile → exec
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10612
diff changeset
   359
            exec(code, scriptlocals)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   360
            if funcname is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   361
                try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   362
                    func = scriptlocals[funcname]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   363
                    self.info('found %s in locals', funcname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   364
                    assert callable(func), '%s (%s) is not callable' % (func, funcname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   365
                except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   366
                    self.critical('no %s in script %s', funcname, migrscript)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   367
                    return None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   368
                return func(*args, **kwargs)
6035
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   369
        elif script_mode == 'sql':
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   370
            from cubicweb.server.sqlutils import sqlexec
f8c7aa251782 [migration] unify process_script command
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 5967
diff changeset
   371
            sqlexec(open(migrscript).read(), self.session.system_sql)
6042
df9cafb8062c [migration] commit only sql script migration
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6035
diff changeset
   372
            self.commit()
3935
2fbb79054a1a imported patch cwctl-shell-textfile
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3812
diff changeset
   373
        else: # script_mode == 'doctest'
2fbb79054a1a imported patch cwctl-shell-textfile
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3812
diff changeset
   374
            import doctest
6687
0b1f5c14646e [shell, testlib] return doc test results in process_script and new assertDocTestFile method on CubicWebTC
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6659
diff changeset
   375
            return doctest.testfile(migrscript, module_relative=False,
0b1f5c14646e [shell, testlib] return doc test results in process_script and new assertDocTestFile method on CubicWebTC
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6659
diff changeset
   376
                                    optionflags=doctest.ELLIPSIS,
6732
f63cef610b97 [migration] improve doctest report format
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6696
diff changeset
   377
                                    # verbose mode when user input is expected
f63cef610b97 [migration] improve doctest report format
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6696
diff changeset
   378
                                    verbose=self.verbosity==2,
f63cef610b97 [migration] improve doctest report format
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6696
diff changeset
   379
                                    report=True,
6687
0b1f5c14646e [shell, testlib] return doc test results in process_script and new assertDocTestFile method on CubicWebTC
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6659
diff changeset
   380
                                    encoding='utf-8',
0b1f5c14646e [shell, testlib] return doc test results in process_script and new assertDocTestFile method on CubicWebTC
Julien Jehannet <julien.jehannet@logilab.fr>
parents: 6659
diff changeset
   381
                                    globs=scriptlocals)
6219
803c1019138e [migration] .pop() is more readable
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6217
diff changeset
   382
        self._context_stack.pop()
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 676
diff changeset
   383
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   384
    def cmd_option_renamed(self, oldname, newname):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
        """a configuration option has been renamed"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   386
        self._option_changes.append(('renamed', oldname, newname))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   387
5580
3e9e6dd54ebb [migration] add migration file commands to follow option group changes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5488
diff changeset
   388
    def cmd_option_group_changed(self, option, oldgroup, newgroup):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   389
        """a configuration option has been moved in another group"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   390
        self._option_changes.append(('moved', option, oldgroup, newgroup))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   391
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   392
    def cmd_option_added(self, optname):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   393
        """a configuration option has been added"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   394
        self._option_changes.append(('added', optname))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   395
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   396
    def cmd_option_removed(self, optname):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   397
        """a configuration option has been removed"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   398
        # can safely be ignored
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   399
        #self._option_changes.append(('removed', optname))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   400
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
    def cmd_option_type_changed(self, optname, oldtype, newvalue):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   402
        """a configuration option's type has changed"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   403
        self._option_changes.append(('typechanged', optname, oldtype, newvalue))
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 676
diff changeset
   404
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   405
    def cmd_add_cubes(self, cubes):
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   406
        """modify the list of used cubes in the in-memory config
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   407
        returns newly inserted cubes, including dependencies
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   408
        """
10612
84468b90e9c1 [py3k] basestring → six.string_types
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   409
        if isinstance(cubes, string_types):
676
270eb87a768a provide a new add_cubes() migration function for cases where the new cubes are linked together by new relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 447
diff changeset
   410
            cubes = (cubes,)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   411
        origcubes = self.config.cubes()
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 676
diff changeset
   412
        newcubes = [p for p in self.config.expand_cubes(cubes)
6221
a5cc5dc15f9d [migration] remove buggy assertion
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6219
diff changeset
   413
                    if not p in origcubes]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   414
        if newcubes:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   415
            self.config.add_cubes(newcubes)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   416
        return newcubes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
10107
1ef92a6193a8 [serverctl] rename remove_cube to drop_cube (closes #4545093)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 10074
diff changeset
   418
    @deprecated('[3.20] use drop_cube() instead of remove_cube()')
2124
5a0b02f37b23 set removedeps to False by default, raise an exception instead of a simple assertion for error, more remove_cube tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2122
diff changeset
   419
    def cmd_remove_cube(self, cube, removedeps=False):
10107
1ef92a6193a8 [serverctl] rename remove_cube to drop_cube (closes #4545093)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 10074
diff changeset
   420
        return self.cmd_drop_cube(cube, removedeps)
1ef92a6193a8 [serverctl] rename remove_cube to drop_cube (closes #4545093)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 10074
diff changeset
   421
1ef92a6193a8 [serverctl] rename remove_cube to drop_cube (closes #4545093)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 10074
diff changeset
   422
    def cmd_drop_cube(self, cube, removedeps=False):
2122
4ea13a828513 add removedeps option to remove_cube to control wether a cube's dependencies should be removed as well or not
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   423
        if removedeps:
4ea13a828513 add removedeps option to remove_cube to control wether a cube's dependencies should be removed as well or not
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   424
            toremove = self.config.expand_cubes([cube])
4ea13a828513 add removedeps option to remove_cube to control wether a cube's dependencies should be removed as well or not
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   425
        else:
4ea13a828513 add removedeps option to remove_cube to control wether a cube's dependencies should be removed as well or not
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   426
            toremove = (cube,)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   427
        origcubes = self.config._cubes
2122
4ea13a828513 add removedeps option to remove_cube to control wether a cube's dependencies should be removed as well or not
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   428
        basecubes = [c for c in origcubes if not c in toremove]
9829
e0e71b45bf8d [migration] don't add new elements to config._cubes in remove_cube
Julien Cristau <julien.cristau@logilab.fr>
parents: 9294
diff changeset
   429
        # don't fake-add any new ones, or we won't be able to really-add them later
e0e71b45bf8d [migration] don't add new elements to config._cubes in remove_cube
Julien Cristau <julien.cristau@logilab.fr>
parents: 9294
diff changeset
   430
        self.config._cubes = tuple(cube for cube in self.config.expand_cubes(basecubes)
e0e71b45bf8d [migration] don't add new elements to config._cubes in remove_cube
Julien Cristau <julien.cristau@logilab.fr>
parents: 9294
diff changeset
   431
                                   if cube in origcubes)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   432
        removed = [p for p in origcubes if not p in self.config._cubes]
8107
e5858a4a1244 [migration] make remove_cube('foo') not fail if foo is already removed. Closes #2106532
Julien Cristau <julien.cristau@logilab.fr>
parents: 7879
diff changeset
   433
        if not cube in removed and cube in origcubes:
2124
5a0b02f37b23 set removedeps to False by default, raise an exception instead of a simple assertion for error, more remove_cube tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2122
diff changeset
   434
            raise ConfigurationError("can't remove cube %s, "
5a0b02f37b23 set removedeps to False by default, raise an exception instead of a simple assertion for error, more remove_cube tests
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2122
diff changeset
   435
                                     "used as a dependency" % cube)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   436
        return removed
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 676
diff changeset
   437
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   438
    def rewrite_configuration(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   439
        configfile = self.config.main_config_file()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   440
        if self._option_changes:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   441
            read_old_config(self.config, self._option_changes, configfile)
3152
7ef7c82daf59 close file handler leak
Aurelien Campeas
parents: 2897
diff changeset
   442
        fd, newconfig = tempfile.mkstemp()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   443
        for optdescr in self._option_changes:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   444
            if optdescr[0] == 'added':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   445
                optdict = self.config.get_option_def(optdescr[1])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   446
                if optdict.get('default') is REQUIRED:
447
0e52d72104a6 pylint fixes
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   447
                    self.config.input_option(optdescr[1], optdict)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   448
        self.config.generate_config(open(newconfig, 'w'))
5487
3ab2682a4b37 [migration] ask_confirm argument of show_diff doesn't contain an _
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5484
diff changeset
   449
        show_diffs(configfile, newconfig, askconfirm=self.confirm is not yes)
3152
7ef7c82daf59 close file handler leak
Aurelien Campeas
parents: 2897
diff changeset
   450
        os.close(fd)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   451
        if exists(newconfig):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   452
            os.unlink(newconfig)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   453
7083
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6732
diff changeset
   454
    # these are overridden by set_log_methods below
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6732
diff changeset
   455
    # only defining here to prevent pylint from complaining
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6732
diff changeset
   456
    info = warning = error = critical = exception = debug = lambda msg,*a,**kw: None
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   457
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   458
from logging import getLogger
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   459
from cubicweb import set_log_methods
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   460
set_log_methods(MigrationHelper, getLogger('cubicweb.migration'))
5027
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   461
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   462
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   463
def version_strictly_lower(a, b):
10720
201028085e12 [migration] NoneType is no longer comparable in py3k
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10662
diff changeset
   464
    if a is None:
201028085e12 [migration] NoneType is no longer comparable in py3k
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10662
diff changeset
   465
        return True
201028085e12 [migration] NoneType is no longer comparable in py3k
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10662
diff changeset
   466
    if b is None:
201028085e12 [migration] NoneType is no longer comparable in py3k
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10662
diff changeset
   467
        return False
5027
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   468
    if a:
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   469
        a = Version(a)
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   470
    if b:
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   471
        b = Version(b)
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   472
    return a < b
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   473
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   474
def max_version(a, b):
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   475
    return str(max(Version(a), Version(b)))
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   476
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   477
class ConfigurationProblem(object):
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   478
    """Each cube has its own list of dependencies on other cubes/versions.
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   479
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   480
    The ConfigurationProblem is used to record the loaded cubes, then to detect
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   481
    inconsistencies in their dependencies.
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   482
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   483
    See configuration management on wikipedia for litterature.
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   484
    """
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   485
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   486
    def __init__(self, config):
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   487
        self.config = config
6657
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   488
        self.cubes = {'cubicweb': cwcfg.cubicweb_version()}
5027
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   489
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   490
    def add_cube(self, name, version):
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   491
        self.cubes[name] = version
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   492
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   493
    def solve(self):
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   494
        self.warnings = []
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   495
        self.errors = []
6657
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   496
        self.dependencies = {}
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   497
        self.reverse_dependencies = {}
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   498
        self.constraints = {}
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   499
        # read dependencies
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   500
        for cube in self.cubes:
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   501
            if cube == 'cubicweb': continue
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   502
            self.dependencies[cube] = dict(self.config.cube_dependencies(cube))
6659
6205927e927a fix problem introduced with previous commit
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6657
diff changeset
   503
            self.dependencies[cube]['cubicweb'] = self.config.cube_depends_cubicweb_version(cube)
6657
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   504
        # compute reverse dependencies
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10615
diff changeset
   505
        for cube, dependencies in self.dependencies.items():
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10615
diff changeset
   506
            for name, constraint in dependencies.items():
6657
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   507
                self.reverse_dependencies.setdefault(name,set())
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   508
                if constraint:
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   509
                    try:
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   510
                        oper, version = constraint.split()
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   511
                        self.reverse_dependencies[name].add( (oper, version, cube) )
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7576
diff changeset
   512
                    except Exception:
6657
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   513
                        self.warnings.append(
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   514
                            'cube %s depends on %s but constraint badly '
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   515
                            'formatted: %s' % (cube, name, constraint))
7537
1af162bd78b8 [migration] bugfix: c-c list was not displaying the name of a cube requiring another cube without constraining the version.
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7083
diff changeset
   516
                else:
1af162bd78b8 [migration] bugfix: c-c list was not displaying the name of a cube requiring another cube without constraining the version.
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7083
diff changeset
   517
                    self.reverse_dependencies[name].add( (None, None, cube) )
6657
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   518
        # check consistency
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   519
        for cube, versions in sorted(self.reverse_dependencies.items()):
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   520
            oper, version, source = None, None, None
5027
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   521
            # simplify constraints
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   522
            if versions:
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   523
                for constraint in versions:
6657
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   524
                    op, ver, src = constraint
5027
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   525
                    if oper is None:
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   526
                        oper = op
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   527
                        version = ver
6657
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   528
                        source = src
5027
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   529
                    elif op == '>=' and oper == '>=':
6657
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   530
                        if version_strictly_lower(version, ver):
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   531
                            version = ver
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   532
                            source = src
7576
1b7fa4df1f83 [cw-ctl] command list should not break when cube depends on version None (closes #1787209)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7537
diff changeset
   533
                    elif op == None:
1b7fa4df1f83 [cw-ctl] command list should not break when cube depends on version None (closes #1787209)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7537
diff changeset
   534
                        continue
5027
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   535
                    else:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   536
                        print('unable to handle %s in %s, set to `%s %s` '
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   537
                              'but currently up to `%s %s`' %
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   538
                              (cube, source, oper, version, op, ver))
5027
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   539
            # "solve" constraint satisfaction problem
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   540
            if cube not in self.cubes:
6657
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   541
                self.errors.append( ('add', cube, version, source) )
5027
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   542
            elif versions:
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   543
                lower_strict = version_strictly_lower(self.cubes[cube], version)
10060
08dc10ef9ab1 Accept '==' operator in cubes dependencies
Christophe de Vienne <christophe@unlish.com>
parents: 9829
diff changeset
   544
                if oper in ('>=','=','=='):
5027
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   545
                    if lower_strict:
6657
402bff898024 [cwctl] command list now checks version of cw, not only versions of cubes
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 6362
diff changeset
   546
                        self.errors.append( ('update', cube, version, source) )
7537
1af162bd78b8 [migration] bugfix: c-c list was not displaying the name of a cube requiring another cube without constraining the version.
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7083
diff changeset
   547
                elif oper is None:
1af162bd78b8 [migration] bugfix: c-c list was not displaying the name of a cube requiring another cube without constraining the version.
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7083
diff changeset
   548
                    pass # no constraint on version
5027
d688daf0a62c [config] move ConfigurationProblem to migration + refactor it to benefit from config methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4721
diff changeset
   549
                else:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10569
diff changeset
   550
                    print('unknown operator', oper)