misc/scripts/repair_file_1-9_migration.py
author Rémi Cardona <remi.cardona@logilab.fr>
Fri, 18 Sep 2015 11:54:12 +0200
changeset 10706 b261d90149d0
parent 10589 7c23b7de2b8d
permissions -rw-r--r--
[server] Port BFSS to py3k The BFSS API changes in python 3: * 'defaultdir' MUST be a unicode object * 'fsencoding' MUST NOT be set In python 2, fsencoding handles both the encoding of file paths on the file system (utf-8 by default, but the system may actually be using something else) and the encoding of file paths that will be stored in the database. So in python 3, we wipe the slate clean: * rely on sys.getfilesystemencoding() to convert unicode objects to bytes * always encode paths to utf-8 for storage in the database Caveat emptor / here be dragons: * sys.getfilesystemencoding() depends on the current locale, which therefore MUST be set properly * when migrating an existing instance from py2 to py3, one MAY need to reencode file paths stored in the database
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
6001
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     1
"""execute this script if you've migration to file >= 1.9.0 with cubicweb <= 3.9.2
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     2
6109
47d9c0e0f7b7 integrate Celso's work on translation file: proper/complete spanish translation, fixed some typos in french translation, occured -> occurred fix in various places
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6001
diff changeset
     3
FYI, this migration occurred :
6001
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     4
* on our intranet on July 07 2010
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     5
* on our extranet on July 16 2010
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     6
"""
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9460
diff changeset
     7
from __future__ import print_function
6001
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     8
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     9
try:
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    10
    backupinstance, = __args__
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    11
except ValueError:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9460
diff changeset
    12
    print('USAGE: cubicweb-ctl shell <instance> repair_file_1-9_migration.py -- <backup instance id>')
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9460
diff changeset
    13
    print()
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9460
diff changeset
    14
    print('you should restored the backup on a new instance, accessible through pyro')
6001
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    15
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    16
from cubicweb import cwconfig, dbapi
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    17
from cubicweb.server.session import hooks_control
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    18
9460
a2a0bc984863 [config] cleanup/refactor server sources file values handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8694
diff changeset
    19
defaultadmin = repo.config.default_admin_config
6001
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    20
backupcfg = cwconfig.instance_configuration(backupinstance)
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    21
backupcfg.repairing = True
9460
a2a0bc984863 [config] cleanup/refactor server sources file values handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8694
diff changeset
    22
backuprepo, backupcnx = dbapi.in_memory_repo_cnx(backupcfg, defaultadmin['login'],
a2a0bc984863 [config] cleanup/refactor server sources file values handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8694
diff changeset
    23
                                                 password=defaultadmin['password'],
7056
51f88f13d6f3 [dbapi] split in_memory_cnx functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6109
diff changeset
    24
                                                 host='localhost')
6001
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    25
backupcu = backupcnx.cursor()
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    26
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    27
with hooks_control(session, session.HOOKS_DENY_ALL):
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    28
    rql('SET X is Y WHERE X is File, Y name "File", NOT X is Y')
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    29
    rql('SET X is_instance_of Y WHERE X is File, Y name "File", NOT X is_instance_of Y')
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    30
    for rtype, in backupcu.execute('DISTINCT Any RTN WHERE X relation_type RT, RT name RTN,'
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    31
                                   'X from_entity Y, Y name "Image", X is CWRelation, '
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    32
                                   'EXISTS(XX is CWRelation, XX relation_type RT, '
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    33
                                   'XX from_entity YY, YY name "File")'):
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    34
        if rtype in ('is', 'is_instance_of'):
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    35
            continue
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9460
diff changeset
    36
        print(rtype)
6001
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    37
        for feid, xeid in backupcu.execute('Any F,X WHERE F %s X, F is IN (File,Image)' % rtype):
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9460
diff changeset
    38
            print('restoring relation %s between file %s and %s' % (rtype, feid, xeid), end=' ')
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9460
diff changeset
    39
            print(rql('SET F %s X WHERE F eid %%(f)s, X eid %%(x)s, NOT F %s X' % (rtype, rtype),
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9460
diff changeset
    40
                      {'f': feid, 'x': xeid}))
6001
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    41
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    42
    for rtype, in backupcu.execute('DISTINCT Any RTN WHERE X relation_type RT, RT name RTN,'
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    43
                                   'X to_entity Y, Y name "Image", X is CWRelation, '
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    44
                                   'EXISTS(XX is CWRelation, XX relation_type RT, '
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    45
                                   'XX to_entity YY, YY name "File")'):
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9460
diff changeset
    46
        print(rtype)
6001
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    47
        for feid, xeid in backupcu.execute('Any F,X WHERE X %s F, F is IN (File,Image)' % rtype):
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9460
diff changeset
    48
            print('restoring relation %s between %s and file %s' % (rtype, xeid, feid), end=' ')
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9460
diff changeset
    49
            print(rql('SET X %s F WHERE F eid %%(f)s, X eid %%(x)s, NOT X %s F' % (rtype, rtype),
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 9460
diff changeset
    50
                      {'f': feid, 'x': xeid}))
6001
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    51
1a21d7d2c959 [migration] add script to repair instances which have lost some relation due to 3.9 + file 1.9 migration (bad Image -> File entity type renaming)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    52
commit()