cubicweb/misc/migration/3.10.9_Any.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Thu, 16 Jun 2016 15:53:01 +0200
changeset 11279 e4f11ef1face
parent 11178 misc/migration/3.10.9_Any.py@b3d3e23be27b
parent 11057 misc/migration/3.10.9_Any.py@0b59724cb3f2
child 12715 d59570643a49
permissions -rw-r--r--
backport 3.22 changes
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
7050
3f2857704444 [3.10.9 migration] don't display progress bar if stdout is not a tty
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7034
diff changeset
     1
import sys
7032
b712477ae286 [config] fix option group clash causing anonymous user configuration to end in the MAIN section instead of the WEB section if cubicweb-dev is installed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7026
diff changeset
     2
7104
62c880e5d980 [migration] explicit first step of 3.10.9 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7050
diff changeset
     3
if confirm('fix some corrupted entities noticed on several instances?'):
62c880e5d980 [migration] explicit first step of 3.10.9 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7050
diff changeset
     4
    rql('DELETE CWConstraint X WHERE NOT E constrained_by X')
62c880e5d980 [migration] explicit first step of 3.10.9 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7050
diff changeset
     5
    rql('SET X is_instance_of Y WHERE X is Y, NOT X is_instance_of Y')
62c880e5d980 [migration] explicit first step of 3.10.9 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7050
diff changeset
     6
    commit()
7034
1ac9715876e3 [migration] cleanup corrupted database, pb noticed on several instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7032
diff changeset
     7
6955
a6c32edabc8d [entity, metadata] huuum, use resolvable url as cwuri... And fix existing ones.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
     8
if confirm('fix existing cwuri?'):
7105
aefd1776122f [migration] improve fix cwuri progress bar
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7104
diff changeset
     9
    from logilab.common.shellutils import progress
6955
a6c32edabc8d [entity, metadata] huuum, use resolvable url as cwuri... And fix existing ones.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    10
    from cubicweb.server.session import hooks_control
a6c32edabc8d [entity, metadata] huuum, use resolvable url as cwuri... And fix existing ones.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
diff changeset
    11
    rset = rql('Any X, XC WHERE X cwuri XC, X cwuri ~= "%/eid/%"')
7550
183a61d1bab9 [3.10] fix typo and error for pre 3.10 migration w/ postgres: we *must* call init_creating else the fti isn't correctly initialized, which may lead to silently rollbacked transaction
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7261
diff changeset
    12
    title = "%i entities to fix" % len(rset)
7105
aefd1776122f [migration] improve fix cwuri progress bar
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7104
diff changeset
    13
    nbops = rset.rowcount
aefd1776122f [migration] improve fix cwuri progress bar
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7104
diff changeset
    14
    enabled = interactive_mode
aefd1776122f [migration] improve fix cwuri progress bar
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7104
diff changeset
    15
    with progress(title=title, nbops=nbops, size=30, enabled=enabled) as pb:
7261
1d1446e9dfe2 place the disable hooks context manager at the right place (closes: #1626694)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7106
diff changeset
    16
        for i,  row in enumerate(rset):
11178
b3d3e23be27b [migration/3.10] fix AttributeError
Julien Cristau <julien.cristau@logilab.fr>
parents: 8694
diff changeset
    17
            with session.deny_all_hooks_but('integrity'):
7106
63f0bc280354 [migration] Lesser memory consumption of cwuri fix + actually commit every 100 processed entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7105
diff changeset
    18
                data = {'eid': row[0], 'cwuri': row[1].replace(u'/eid', u'')}
63f0bc280354 [migration] Lesser memory consumption of cwuri fix + actually commit every 100 processed entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7105
diff changeset
    19
                rql('SET X cwuri %(cwuri)s WHERE X eid %(eid)s', data)
7261
1d1446e9dfe2 place the disable hooks context manager at the right place (closes: #1626694)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7106
diff changeset
    20
            if not i % 100: # commit every 100 entities to limit memory consumption
1d1446e9dfe2 place the disable hooks context manager at the right place (closes: #1626694)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7106
diff changeset
    21
                pb.text = "%i committed" % i
1d1446e9dfe2 place the disable hooks context manager at the right place (closes: #1626694)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7106
diff changeset
    22
                commit(ask_confirm=False)
1d1446e9dfe2 place the disable hooks context manager at the right place (closes: #1626694)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7106
diff changeset
    23
            pb.update()
7105
aefd1776122f [migration] improve fix cwuri progress bar
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7104
diff changeset
    24
        commit(ask_confirm=False)
7032
b712477ae286 [config] fix option group clash causing anonymous user configuration to end in the MAIN section instead of the WEB section if cubicweb-dev is installed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7026
diff changeset
    25
b712477ae286 [config] fix option group clash causing anonymous user configuration to end in the MAIN section instead of the WEB section if cubicweb-dev is installed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7026
diff changeset
    26
try:
b712477ae286 [config] fix option group clash causing anonymous user configuration to end in the MAIN section instead of the WEB section if cubicweb-dev is installed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7026
diff changeset
    27
    from cubicweb import devtools
b712477ae286 [config] fix option group clash causing anonymous user configuration to end in the MAIN section instead of the WEB section if cubicweb-dev is installed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7026
diff changeset
    28
    option_group_changed('anonymous-user', 'main', 'web')
b712477ae286 [config] fix option group clash causing anonymous user configuration to end in the MAIN section instead of the WEB section if cubicweb-dev is installed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7026
diff changeset
    29
    option_group_changed('anonymous-password', 'main', 'web')
b712477ae286 [config] fix option group clash causing anonymous user configuration to end in the MAIN section instead of the WEB section if cubicweb-dev is installed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7026
diff changeset
    30
except ImportError:
b712477ae286 [config] fix option group clash causing anonymous user configuration to end in the MAIN section instead of the WEB section if cubicweb-dev is installed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7026
diff changeset
    31
    # cubicweb-dev unavailable, nothing needed
b712477ae286 [config] fix option group clash causing anonymous user configuration to end in the MAIN section instead of the WEB section if cubicweb-dev is installed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7026
diff changeset
    32
    pass