cubicweb/misc/scripts/migration_helper.py
author Denis Laxalde <denis.laxalde@logilab.fr>
Fri, 05 Apr 2019 17:58:19 +0200
changeset 12567 26744ad37953
parent 11767 432f87a63057
permissions -rw-r--r--
Drop python2 support This mostly consists in removing the dependency on "six" and updating the code to use only Python3 idioms. Notice that we previously used TemporaryDirectory from cubicweb.devtools.testlib for compatibility with Python2. We now directly import it from tempfile.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
11185
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     1
# copyright 2014 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     3
#
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     4
# This file is part of CubicWeb.
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     5
#
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     9
# any later version.
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    10
#
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    14
# details.
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    15
#
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    18
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    19
"""Helper functions for migrations that aren't reliable enough or too dangerous
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    20
to be available in the standard migration environment
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    21
"""
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    22
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    23
def drop_entity_types_fast(*etypes, **kwargs):
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    24
    """drop an entity type bypassing all hooks
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    25
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    26
    here be dragons.
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    27
    """
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    28
    # XXX cascade deletion through composite relations?
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    29
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    30
    for etype in etypes:
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    31
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    32
        if etype not in schema:
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    33
            print('%s does not exist' % etype)
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    34
            continue
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    35
        etype = schema[etype]
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    36
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    37
        # ignore attributes and inlined rels since they'll be dropped anyway
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    38
        srels = [x.type for x in etype.subject_relations() if x.eid and not (x.final or x.inlined)]
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    39
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    40
        orels = [x.type for x in etype.object_relations() if x.eid and not x.inlined]
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    41
        inlined_rels = [x for x in etype.object_relations() if x.eid and x.inlined]
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    42
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    43
        # eids to be deleted could be listed in some other entity tables through inlined relations
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    44
        for rtype in inlined_rels:
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    45
            for subjtype in rtype.subjects(etype):
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    46
                if subjtype in etypes:
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    47
                    continue
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    48
                sql('UPDATE cw_%(stype)s SET cw_%(rtype)s = NULL '
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    49
                    'WHERE cw_%(rtype)s IN (SELECT eid FROM entities WHERE type = %%s)' %
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    50
                    {'stype': subjtype.type, 'rtype': rtype.type},
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    51
                    (etype.type,))
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    52
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    53
        for rel in srels:
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    54
            if all(subj in etypes for subj in rel.subjects()) or all(obj in etypes for obj in rel.objects()):
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    55
                sql('DELETE FROM %s_relation' % rel.type)
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    56
            else:
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    57
                sql('DELETE FROM %s_relation WHERE eid_from IN (SELECT eid FROM entities WHERE type = %%s)' % rel.type, (etype.type,))
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    58
        for rel in orels:
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    59
            if all(subj in etypes for subj in rel.subjects()) or all(obj in etypes for obj in rel.objects()):
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    60
                sql('DELETE FROM %s_relation' % rel.type)
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    61
            else:
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    62
                sql('DELETE FROM %s_relation WHERE eid_to IN (SELECT eid FROM entities WHERE type = %%s)' % rel, (etype.type,))
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    63
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    64
        sql('DELETE FROM appears WHERE uid IN (SELECT eid FROM entities WHERE type = %s)', (etype.type,))
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    65
        sql('DELETE FROM cw_%s' % etype.type)
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    66
        sql('DELETE FROM entities WHERE type = %s', (etype.type,))
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    67
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    68
    for etype in etypes:
40e74d6d7e99 Add migration helper function to delete entity types faster
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    69
        drop_entity_type(etype, **kwargs)