misc/migration/bootstrapmigration_repository.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Tue, 22 Dec 2009 21:02:37 +0100
branchstable
changeset 4195 86dcaf6bb92f
parent 3788 3d5abcd7c852
child 3890 d7a270f50f54
child 4212 ab6573088b4a
permissions -rw-r--r--
closes #601987 1) sqlutils.restore_from_file have to use its confirm argument when a command fail, to propose to continue there (this can't be handled by the caller) 2) source.restore method hence needs to take this confirmation callback as argument 3) properly fix places where 'drop' was given instead of 'confirm'
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     1
"""allways executed before all others in server migration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     2
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     3
it should only include low level schema changes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     4
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     5
:organization: Logilab
1977
606923dff11b big bunch of copyright / docstring update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1954
diff changeset
     6
:copyright: 2001-2009 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     7
:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr
1977
606923dff11b big bunch of copyright / docstring update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1954
diff changeset
     8
:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     9
"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    10
2275
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    11
applcubicwebversion, cubicwebversion = versions_map['cubicweb']
bc0bed0616a3 fix #344387, remember upgraded version step by step
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    12
2463
5200c0f7d2d5 add migration script for the new cwuri metadata attribute
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2462
diff changeset
    13
if applcubicwebversion < (3, 4, 0) and cubicwebversion >= (3, 4, 0):
2480
a04caff1322e take care of sources with no write support (eg ldapuser)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2463
diff changeset
    14
    from cubicweb import RepositoryError
2463
5200c0f7d2d5 add migration script for the new cwuri metadata attribute
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2462
diff changeset
    15
    from cubicweb.server.hooks import uniquecstrcheck_before_modification
5200c0f7d2d5 add migration script for the new cwuri metadata attribute
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2462
diff changeset
    16
    session.set_shared_data('do-not-insert-cwuri', True)
5200c0f7d2d5 add migration script for the new cwuri metadata attribute
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2462
diff changeset
    17
    repo.hm.unregister_hook(uniquecstrcheck_before_modification, 'before_add_entity', '')
5200c0f7d2d5 add migration script for the new cwuri metadata attribute
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2462
diff changeset
    18
    repo.hm.unregister_hook(uniquecstrcheck_before_modification, 'before_update_entity', '')
5200c0f7d2d5 add migration script for the new cwuri metadata attribute
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2462
diff changeset
    19
    add_relation_type('cwuri')
5200c0f7d2d5 add migration script for the new cwuri metadata attribute
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2462
diff changeset
    20
    base_url = session.base_url()
5200c0f7d2d5 add migration script for the new cwuri metadata attribute
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2462
diff changeset
    21
    # use an internal session since some entity might forbid modifications to admin
5200c0f7d2d5 add migration script for the new cwuri metadata attribute
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2462
diff changeset
    22
    isession = repo.internal_session()
5200c0f7d2d5 add migration script for the new cwuri metadata attribute
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2462
diff changeset
    23
    for eid, in rql('Any X', ask_confirm=False):
2696
b45d852990d5 [3.4 migration] only set cwuri for entities from the system source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2578
diff changeset
    24
        type, source, extid = session.describe(eid)
b45d852990d5 [3.4 migration] only set cwuri for entities from the system source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2578
diff changeset
    25
        if source == 'system':
2480
a04caff1322e take care of sources with no write support (eg ldapuser)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2463
diff changeset
    26
            isession.execute('SET X cwuri %(u)s WHERE X eid %(x)s',
a04caff1322e take care of sources with no write support (eg ldapuser)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2463
diff changeset
    27
                             {'x': eid, 'u': base_url + u'eid/%s' % eid})
2463
5200c0f7d2d5 add migration script for the new cwuri metadata attribute
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2462
diff changeset
    28
    isession.commit()
5200c0f7d2d5 add migration script for the new cwuri metadata attribute
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2462
diff changeset
    29
    repo.hm.register_hook(uniquecstrcheck_before_modification, 'before_add_entity', '')
5200c0f7d2d5 add migration script for the new cwuri metadata attribute
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2462
diff changeset
    30
    repo.hm.register_hook(uniquecstrcheck_before_modification, 'before_update_entity', '')
2578
b717ebef04d8 [migration] reset do-not-insert-cwuri once it's properly in the schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2480
diff changeset
    31
    session.set_shared_data('do-not-insert-cwuri', False)
2463
5200c0f7d2d5 add migration script for the new cwuri metadata attribute
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2462
diff changeset
    32
2965
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    33
if applcubicwebversion < (3, 5, 0) and cubicwebversion >= (3, 5, 0):
3788
3d5abcd7c852 [migration] warn if 3.5 workflow migration is going to fail
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3305
diff changeset
    34
    # check that migration is not doomed
3d5abcd7c852 [migration] warn if 3.5 workflow migration is going to fail
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3305
diff changeset
    35
    rset = rql('Any X,Y WHERE X transition_of E, Y transition_of E, '
3d5abcd7c852 [migration] warn if 3.5 workflow migration is going to fail
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3305
diff changeset
    36
               'X name N, Y name N, NOT X identity Y',
3d5abcd7c852 [migration] warn if 3.5 workflow migration is going to fail
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3305
diff changeset
    37
               ask_confirm=False)
3d5abcd7c852 [migration] warn if 3.5 workflow migration is going to fail
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3305
diff changeset
    38
    if rset:
3d5abcd7c852 [migration] warn if 3.5 workflow migration is going to fail
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3305
diff changeset
    39
        from logilab.common.shellutils import ASK
3d5abcd7c852 [migration] warn if 3.5 workflow migration is going to fail
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3305
diff changeset
    40
        if not ASK.confirm('Migration will fail because of transitions with the same name. '
3d5abcd7c852 [migration] warn if 3.5 workflow migration is going to fail
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3305
diff changeset
    41
                           'Continue anyway ?'):
3d5abcd7c852 [migration] warn if 3.5 workflow migration is going to fail
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3305
diff changeset
    42
            import sys
3d5abcd7c852 [migration] warn if 3.5 workflow migration is going to fail
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3305
diff changeset
    43
            sys.exit(1)
3d5abcd7c852 [migration] warn if 3.5 workflow migration is going to fail
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 3305
diff changeset
    44
    # proceed with migration
2965
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    45
    add_entity_type('Workflow')
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    46
    add_entity_type('BaseTransition')
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    47
    add_entity_type('WorkflowTransition')
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    48
    add_entity_type('SubWorkflowExitPoint')
3218
2a4bbe3fa4f3 [3.5 migration] avoid removing defined transitions...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2965
diff changeset
    49
    # drop explicit 'State allowed_transition Transition' since it should be
2a4bbe3fa4f3 [3.5 migration] avoid removing defined transitions...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2965
diff changeset
    50
    # infered due to yams inheritance.  However we've to disable the schema
2a4bbe3fa4f3 [3.5 migration] avoid removing defined transitions...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2965
diff changeset
    51
    # sync hook first to avoid to destroy existing data...
2a4bbe3fa4f3 [3.5 migration] avoid removing defined transitions...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2965
diff changeset
    52
    from cubicweb.server.schemahooks import after_del_relation_type
2a4bbe3fa4f3 [3.5 migration] avoid removing defined transitions...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2965
diff changeset
    53
    repo.hm.unregister_hook(after_del_relation_type,
2a4bbe3fa4f3 [3.5 migration] avoid removing defined transitions...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2965
diff changeset
    54
                            'after_delete_relation', 'relation_type')
2a4bbe3fa4f3 [3.5 migration] avoid removing defined transitions...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2965
diff changeset
    55
    try:
2a4bbe3fa4f3 [3.5 migration] avoid removing defined transitions...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2965
diff changeset
    56
        drop_relation_definition('State', 'allowed_transition', 'Transition')
2a4bbe3fa4f3 [3.5 migration] avoid removing defined transitions...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2965
diff changeset
    57
    finally:
2a4bbe3fa4f3 [3.5 migration] avoid removing defined transitions...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2965
diff changeset
    58
        repo.hm.register_hook(after_del_relation_type,
2a4bbe3fa4f3 [3.5 migration] avoid removing defined transitions...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2965
diff changeset
    59
                              'after_delete_relation', 'relation_type')
2965
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    60
    schema.rebuild_infered_relations() # need to be explicitly called once everything is in place
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    61
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    62
    for et in rql('DISTINCT Any ET,ETN WHERE S state_of ET, ET name ETN',
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    63
                  ask_confirm=False).entities():
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    64
        wf = add_workflow(u'default %s workflow' % et.name, et.name,
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    65
                          ask_confirm=False)
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    66
        rql('SET S state_of WF WHERE S state_of ET, ET eid %(et)s, WF eid %(wf)s',
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    67
            {'et': et.eid, 'wf': wf.eid}, 'et', ask_confirm=False)
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    68
        rql('SET T transition_of WF WHERE T transition_of ET, ET eid %(et)s, WF eid %(wf)s',
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    69
            {'et': et.eid, 'wf': wf.eid}, 'et', ask_confirm=False)
3305
abae9045d2e4 [migration] fix initial_state migration: at this point, S state_of ET is no more true (but we don't need it...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3218
diff changeset
    70
        rql('SET WF initial_state S WHERE ET initial_state S, ET eid %(et)s, WF eid %(wf)s',
2965
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    71
            {'et': et.eid, 'wf': wf.eid}, 'et', ask_confirm=False)
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    72
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    73
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    74
    rql('DELETE TrInfo TI WHERE NOT TI from_state S')
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    75
    rql('SET TI by_transition T WHERE TI from_state FS, TI to_state TS, '
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    76
        'FS allowed_transition T, T destination_state TS')
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    77
    checkpoint()
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    78
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    79
    drop_relation_definition('State', 'state_of', 'CWEType')
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    80
    drop_relation_definition('Transition', 'transition_of', 'CWEType')
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    81
    drop_relation_definition('CWEType', 'initial_state', 'State')
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    82
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    83
    sync_schema_props_perms()
75ad5cb00b88 3.5 migration script
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
    84
1952
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1501
diff changeset
    85
if applcubicwebversion < (3, 2, 2) and cubicwebversion >= (3, 2, 1):
2462
9e670072884d fix indentation problems in bootstrap_migration (use 4 spaces instead of 3)
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2275
diff changeset
    86
    from base64 import b64encode
9e670072884d fix indentation problems in bootstrap_migration (use 4 spaces instead of 3)
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2275
diff changeset
    87
    for table in ('entities', 'deleted_entities'):
9e670072884d fix indentation problems in bootstrap_migration (use 4 spaces instead of 3)
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2275
diff changeset
    88
        for eid, extid in sql('SELECT eid, extid FROM %s WHERE extid is NOT NULL'
9e670072884d fix indentation problems in bootstrap_migration (use 4 spaces instead of 3)
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2275
diff changeset
    89
                              % table, ask_confirm=False):
9e670072884d fix indentation problems in bootstrap_migration (use 4 spaces instead of 3)
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2275
diff changeset
    90
            sql('UPDATE %s SET extid=%%(extid)s WHERE eid=%%(eid)s' % table,
9e670072884d fix indentation problems in bootstrap_migration (use 4 spaces instead of 3)
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2275
diff changeset
    91
                {'extid': b64encode(extid), 'eid': eid}, ask_confirm=False)
9e670072884d fix indentation problems in bootstrap_migration (use 4 spaces instead of 3)
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2275
diff changeset
    92
    checkpoint()
1952
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1501
diff changeset
    93
1501
c80ca83a382f migrate card cube in bootstrap migration
sylvain.thenault@logilab.fr
parents: 1399
diff changeset
    94
if applcubicwebversion < (3, 2, 0) and cubicwebversion >= (3, 2, 0):
2462
9e670072884d fix indentation problems in bootstrap_migration (use 4 spaces instead of 3)
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 2275
diff changeset
    95
    add_cube('card', update_database=False)