server/schemaserial.py
author Julien Cristau <julien.cristau@logilab.fr>
Fri, 24 Jul 2015 09:57:08 +0200
changeset 10644 c43e5dc41f8b
parent 10553 1d824df4f2bd
child 10589 7c23b7de2b8d
permissions -rw-r--r--
[devtools] add has_cache for postgres (closes #5739624) devtools stores info about existing dbs in the db handler, but in the case of postgresql that doesn't take into account the path to the cluster's datadir. Which means if we run two test modules (in the same test run), we'll create a "__default_empty_db__" for the first one, cache its existence, and then when moving on to the other module, believe the template already exists (but since the datadir depends on the test module's path, it does not). This patch is a bit of a kludge, and it would be better to make the cache key include enough data to not need this, but I'm not sure how to do that.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
8945
ba9e3fbfa5a5 [schemaserial] serialize additional yams parameter for customs type
Vincent Michel <vincent.michel@logilab.fr>
parents: 8748
diff changeset
     1
# copyright 2003-2013 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
5896
67683b7e591a cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5558
diff changeset
    18
"""functions for schema / permissions (de)serialization using RQL"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
3854
8633cd05b6b5 no progress bar in apycot environment
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    22
import os
8945
ba9e3fbfa5a5 [schemaserial] serialize additional yams parameter for customs type
Vincent Michel <vincent.michel@logilab.fr>
parents: 8748
diff changeset
    23
import json
10375
28ec01db78b3 [server] Do not use progress bar when stdout is not a TTY
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 10191
diff changeset
    24
import sys
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
10375
28ec01db78b3 [server] Do not use progress bar when stdout is not a TTY
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 10191
diff changeset
    26
from logilab.common.shellutils import ProgressBar, DummyProgressBar
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
10200
cceb2c7c02f4 Use our version of schema2sql
Julien Cristau <julien.cristau@logilab.fr>
parents: 10191
diff changeset
    28
from yams import BadSchemaDefinition, schema as schemamod, buildobjs as ybo
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    29
9468
39b7a91a3f4c [repo] pylint cleanup, mainly of imports, with a bit of style
Julien Cristau <julien.cristau@logilab.fr>
parents: 9463
diff changeset
    30
from cubicweb import Binary
8945
ba9e3fbfa5a5 [schemaserial] serialize additional yams parameter for customs type
Vincent Michel <vincent.michel@logilab.fr>
parents: 8748
diff changeset
    31
from cubicweb.schema import (KNOWN_RPROPERTIES, CONSTRAINTS, ETYPE_NAME_MAP,
9468
39b7a91a3f4c [repo] pylint cleanup, mainly of imports, with a bit of style
Julien Cristau <julien.cristau@logilab.fr>
parents: 9463
diff changeset
    32
                             VIRTUAL_RTYPES)
10200
cceb2c7c02f4 Use our version of schema2sql
Julien Cristau <julien.cristau@logilab.fr>
parents: 10191
diff changeset
    33
from cubicweb.server import sqlutils, schema2sql as y2sql
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    34
8945
ba9e3fbfa5a5 [schemaserial] serialize additional yams parameter for customs type
Vincent Michel <vincent.michel@logilab.fr>
parents: 8748
diff changeset
    35
9503
562cef272a35 [schemaserial] Replace 'cursor' with 'cnx'
Julien Cristau <julien.cristau@logilab.fr>
parents: 9490
diff changeset
    36
def group_mapping(cnx, interactive=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    37
    """create a group mapping from an rql cursor
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    38
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    39
    A group mapping has standard group names as key (managers, owners at least)
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
    40
    and the actual CWGroup entity's eid as associated value.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    41
    In interactive mode (the default), missing groups'eid will be prompted
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    42
    from the user.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    43
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    44
    res = {}
9503
562cef272a35 [schemaserial] Replace 'cursor' with 'cnx'
Julien Cristau <julien.cristau@logilab.fr>
parents: 9490
diff changeset
    45
    for eid, name in cnx.execute('Any G, N WHERE G is CWGroup, G name N',
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    46
                                    build_descr=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    47
        res[name] = eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    48
    if not interactive:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    49
        return res
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
    missing = [g for g in ('owners', 'managers', 'users', 'guests') if not g in res]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    51
    if missing:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
        print 'some native groups are missing but the following groups have been found:'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    53
        print '\n'.join('* %s (%s)' % (n, eid) for n, eid in res.items())
1630
41aadba8b29f delete-trailing-whitespaces, check table existance
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
    54
        print
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    55
        print 'enter the eid of a to group to map to each missing native group'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    56
        print 'or just type enter to skip permissions granted to a group'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    57
        for group in missing:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    58
            while True:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    59
                value = raw_input('eid for group %s: ' % group).strip()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    60
                if not value:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    61
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    62
                try:
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8695
diff changeset
    63
                    eid = int(value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    64
                except ValueError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    65
                    print 'eid should be an integer'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    66
                    continue
5970
fb8acdab4e12 [broken schema] fix the group_mapping function in case where some standard group isn't found: we currently loop for ever, boring the user by asking ever and ever the same question...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5896
diff changeset
    67
                for eid_ in res.values():
fb8acdab4e12 [broken schema] fix the group_mapping function in case where some standard group isn't found: we currently loop for ever, boring the user by asking ever and ever the same question...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5896
diff changeset
    68
                    if eid == eid_:
fb8acdab4e12 [broken schema] fix the group_mapping function in case where some standard group isn't found: we currently loop for ever, boring the user by asking ever and ever the same question...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5896
diff changeset
    69
                        break
fb8acdab4e12 [broken schema] fix the group_mapping function in case where some standard group isn't found: we currently loop for ever, boring the user by asking ever and ever the same question...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5896
diff changeset
    70
                else:
fb8acdab4e12 [broken schema] fix the group_mapping function in case where some standard group isn't found: we currently loop for ever, boring the user by asking ever and ever the same question...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5896
diff changeset
    71
                    print 'eid is not a group eid'
fb8acdab4e12 [broken schema] fix the group_mapping function in case where some standard group isn't found: we currently loop for ever, boring the user by asking ever and ever the same question...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5896
diff changeset
    72
                    continue
fb8acdab4e12 [broken schema] fix the group_mapping function in case where some standard group isn't found: we currently loop for ever, boring the user by asking ever and ever the same question...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5896
diff changeset
    73
                res[name] = eid
fb8acdab4e12 [broken schema] fix the group_mapping function in case where some standard group isn't found: we currently loop for ever, boring the user by asking ever and ever the same question...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5896
diff changeset
    74
                break
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    75
    return res
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    76
9489
0581f6d2812e [server/schemaserial] rename cursor argument to cnx
Julien Cristau <julien.cristau@logilab.fr>
parents: 9468
diff changeset
    77
def cstrtype_mapping(cnx):
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
    78
    """cached constraint types mapping"""
9489
0581f6d2812e [server/schemaserial] rename cursor argument to cnx
Julien Cristau <julien.cristau@logilab.fr>
parents: 9468
diff changeset
    79
    map = dict(cnx.execute('Any T, X WHERE X is CWConstraintType, X name T'))
5379
c082dea0731b [schema] update for yams 0.29: BoundConstraint renamed to BoundaryConstraint; new constraint messages; ObjectRelation deprecated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
    80
    return map
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
    81
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    82
# schema / perms deserialization ##############################################
4791
b9a1b16a8d26 fix schema serialization, bad repartition of changes between stable and default
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4789
diff changeset
    83
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
    84
def deserialize_schema(schema, cnx):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    85
    """return a schema according to information stored in an rql database
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
    86
    as CWRType and CWEType entities
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
    """
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
    88
    repo = cnx.repo
2044
d5589c5bc4e6 alter table in schema serial when a renamed type is detected
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    89
    dbhelper = repo.system_source.dbhelper
9956
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
    90
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
    91
    # Computed Rtype
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
    92
    with cnx.ensure_cnx_set:
10191
cc4b8021bdf3 [schemaserial] fix detection of CWComputedRType table, closes #4901150
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10115
diff changeset
    93
        tables = set(t.lower() for t in dbhelper.list_tables(cnx.cnxset.cu))
cc4b8021bdf3 [schemaserial] fix detection of CWComputedRType table, closes #4901150
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10115
diff changeset
    94
        has_computed_relations = 'cw_cwcomputedrtype' in tables
9968
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
    95
    # computed attribute
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
    96
    try:
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
    97
        cnx.system_sql("SELECT cw_formula FROM cw_CWAttribute")
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
    98
        has_computed_attributes = True
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
    99
    except Exception:
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   100
        cnx.rollback()
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   101
        has_computed_attributes = False
9956
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   102
6191
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   103
    # XXX bw compat (3.6 migration)
10365
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10200
diff changeset
   104
    sqlcu = cnx.system_sql("SELECT * FROM cw_CWRType WHERE cw_name='symetric'")
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10200
diff changeset
   105
    if sqlcu.fetchall():
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10200
diff changeset
   106
        sql = dbhelper.sql_rename_col('cw_CWRType', 'cw_symetric', 'cw_symmetric',
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10200
diff changeset
   107
                                      dbhelper.TYPE_MAPPING['Boolean'], True)
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10200
diff changeset
   108
        sqlcu.execute(sql)
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10200
diff changeset
   109
        sqlcu.execute("UPDATE cw_CWRType SET cw_name='symmetric' WHERE cw_name='symetric'")
21461f80f348 [connection] remove ensure_cnx_set context manager uses
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10200
diff changeset
   110
        cnx.commit()
6191
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   111
    ertidx = {}
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   112
    copiedeids = set()
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   113
    permsidx = deserialize_ertype_permissions(cnx)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   114
    schema.reading_from_database = True
9959
6047bf8df9e5 [schema serial] add some comments
Laura Médioni <laura.medioni@logilab.fr>
parents: 9956
diff changeset
   115
    # load every entity types
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   116
    for eid, etype, desc in cnx.execute(
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   117
        'Any X, N, D WHERE X is CWEType, X name N, X description D',
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   118
        build_descr=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   119
        # base types are already in the schema, skip them
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   120
        if etype in schemamod.BASE_TYPES:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   121
            # just set the eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   122
            eschema = schema.eschema(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   123
            eschema.eid = eid
6191
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   124
            ertidx[eid] = etype
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   125
            continue
2044
d5589c5bc4e6 alter table in schema serial when a renamed type is detected
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   126
        if etype in ETYPE_NAME_MAP:
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   127
            needcopy = False
2044
d5589c5bc4e6 alter table in schema serial when a renamed type is detected
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   128
            netype = ETYPE_NAME_MAP[etype]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
            # can't use write rql queries at this point, use raw sql
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   130
            sqlexec = cnx.system_sql
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   131
            if sqlexec('SELECT 1 FROM %(p)sCWEType WHERE %(p)sname=%%(n)s'
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   132
                       % {'p': sqlutils.SQL_PREFIX}, {'n': netype}).fetchone():
6191
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   133
                # the new type already exists, we should copy (eg make existing
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   134
                # instances of the old type instances of the new type)
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   135
                assert etype.lower() != netype.lower()
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   136
                needcopy = True
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   137
            else:
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   138
                # the new type doesn't exist, we should rename
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   139
                sqlexec('UPDATE %(p)sCWEType SET %(p)sname=%%(n)s WHERE %(p)seid=%%(x)s'
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   140
                        % {'p': sqlutils.SQL_PREFIX}, {'x': eid, 'n': netype})
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   141
                if etype.lower() != netype.lower():
6734
ec9a5efdc451 make rename_table work with MS SQL (needs updated logilab.database)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6213
diff changeset
   142
                    alter_table_sql = dbhelper.sql_rename_table(sqlutils.SQL_PREFIX+etype,
ec9a5efdc451 make rename_table work with MS SQL (needs updated logilab.database)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6213
diff changeset
   143
                                                                sqlutils.SQL_PREFIX+netype)
ec9a5efdc451 make rename_table work with MS SQL (needs updated logilab.database)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6213
diff changeset
   144
                    sqlexec(alter_table_sql)
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   145
            sqlexec('UPDATE entities SET type=%(n)s WHERE type=%(x)s',
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   146
                    {'x': etype, 'n': netype})
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   147
            cnx.commit(False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   148
            tocleanup = [eid]
7543
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   149
            tocleanup += (eid for eid, cached in repo._type_source_cache.iteritems()
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
   150
                          if etype == cached[0])
1630
41aadba8b29f delete-trailing-whitespaces, check table existance
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   151
            repo.clear_caches(tocleanup)
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   152
            cnx.commit(False)
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   153
            if needcopy:
6191
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   154
                ertidx[eid] = netype
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   155
                copiedeids.add(eid)
5558
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   156
                # copy / CWEType entity removal expected to be done through
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   157
                # rename_entity_type in a migration script
afd1face1faf [schema migration] make some stuff to ease file 1.9 migration : we want to kill the Image entity so that existing image are turned into (existing entity type) File entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   158
                continue
2044
d5589c5bc4e6 alter table in schema serial when a renamed type is detected
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   159
            etype = netype
6191
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   160
        ertidx[eid] = etype
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   161
        eschema = schema.add_entity_type(
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   162
            ybo.EntityType(name=etype, description=desc, eid=eid))
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   163
        set_perms(eschema, permsidx)
9959
6047bf8df9e5 [schema serial] add some comments
Laura Médioni <laura.medioni@logilab.fr>
parents: 9956
diff changeset
   164
    # load inheritance relations
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   165
    for etype, stype in cnx.execute(
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   166
        'Any XN, ETN WHERE X is CWEType, X name XN, X specializes ET, ET name ETN',
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   167
        build_descr=False):
6080
d0cb8fde4957 [schema] consider ETYPE_NAME_MAP when deserializing specialization relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5970
diff changeset
   168
        etype = ETYPE_NAME_MAP.get(etype, etype)
d0cb8fde4957 [schema] consider ETYPE_NAME_MAP when deserializing specialization relations
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 5970
diff changeset
   169
        stype = ETYPE_NAME_MAP.get(stype, stype)
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   170
        schema.eschema(etype)._specialized_type = stype
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   171
        schema.eschema(stype)._specialized_by.append(etype)
10553
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10411
diff changeset
   172
    if has_computed_relations:
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10411
diff changeset
   173
        rset = cnx.execute(
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10411
diff changeset
   174
            'Any X, N, R, D WHERE X is CWComputedRType, X name N, '
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10411
diff changeset
   175
            'X rule R, X description D')
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10411
diff changeset
   176
        for eid, rule_name, rule, description in rset.rows:
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10411
diff changeset
   177
            rtype = ybo.ComputedRelation(name=rule_name, rule=rule, eid=eid,
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10411
diff changeset
   178
                                         description=description)
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10411
diff changeset
   179
            rschema = schema.add_relation_type(rtype)
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10411
diff changeset
   180
            set_perms(rschema, permsidx)
9959
6047bf8df9e5 [schema serial] add some comments
Laura Médioni <laura.medioni@logilab.fr>
parents: 9956
diff changeset
   181
    # load every relation types
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   182
    for eid, rtype, desc, sym, il, ftc in cnx.execute(
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   183
        'Any X,N,D,S,I,FTC WHERE X is CWRType, X name N, X description D, '
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4312
diff changeset
   184
        'X symmetric S, X inlined I, X fulltext_container FTC', build_descr=False):
6191
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   185
        ertidx[eid] = rtype
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   186
        rschema = schema.add_relation_type(
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   187
            ybo.RelationType(name=rtype, description=desc,
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   188
                             symmetric=bool(sym), inlined=bool(il),
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   189
                             fulltext_container=ftc, eid=eid))
9959
6047bf8df9e5 [schema serial] add some comments
Laura Médioni <laura.medioni@logilab.fr>
parents: 9956
diff changeset
   190
    # remains to load every relation definitions (ie relations and attributes)
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   191
    cstrsidx = deserialize_rdef_constraints(cnx)
6191
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   192
    pendingrdefs = []
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   193
    # closure to factorize common code of attribute/relation rdef addition
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   194
    def _add_rdef(rdefeid, seid, reid, oeid, **kwargs):
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   195
        rdef = ybo.RelationDefinition(ertidx[seid], ertidx[reid], ertidx[oeid],
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   196
                                      constraints=cstrsidx.get(rdefeid, ()),
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   197
                                      eid=rdefeid, **kwargs)
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   198
        if seid in copiedeids or oeid in copiedeids:
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   199
            # delay addition of this rdef. We'll insert them later if needed. We
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   200
            # have to do this because:
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   201
            #
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   202
            # * on etype renaming, we want relation of the old entity type being
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   203
            #   redirected to the new type during migration
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   204
            #
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   205
            # * in the case of a copy, we've to take care that rdef already
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   206
            #   existing in the schema are not overwritten by a redirected one,
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   207
            #   since we want correct eid on them (redirected rdef will be
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   208
            #   removed in rename_entity_type)
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   209
            pendingrdefs.append(rdef)
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   210
        else:
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   211
            # add_relation_def return a RelationDefinitionSchema if it has been
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   212
            # actually added (can be None on duplicated relation definitions,
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   213
            # e.g. if the relation type is marked as beeing symmetric)
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   214
            rdefs = schema.add_relation_def(rdef)
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   215
            if rdefs is not None:
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6199
diff changeset
   216
                ertidx[rdefeid] = rdefs
6191
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   217
                set_perms(rdefs, permsidx)
8945
ba9e3fbfa5a5 [schemaserial] serialize additional yams parameter for customs type
Vincent Michel <vincent.michel@logilab.fr>
parents: 8748
diff changeset
   218
    # Get the type parameters for additional base types.
ba9e3fbfa5a5 [schemaserial] serialize additional yams parameter for customs type
Vincent Michel <vincent.michel@logilab.fr>
parents: 8748
diff changeset
   219
    try:
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   220
        extra_props = dict(cnx.execute('Any X, XTP WHERE X is CWAttribute, '
9959
6047bf8df9e5 [schema serial] add some comments
Laura Médioni <laura.medioni@logilab.fr>
parents: 9956
diff changeset
   221
                                       'X extra_props XTP'))
8945
ba9e3fbfa5a5 [schemaserial] serialize additional yams parameter for customs type
Vincent Michel <vincent.michel@logilab.fr>
parents: 8748
diff changeset
   222
    except Exception:
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   223
        cnx.critical('Previous CRITICAL notification about extra_props is not '
9959
6047bf8df9e5 [schema serial] add some comments
Laura Médioni <laura.medioni@logilab.fr>
parents: 9956
diff changeset
   224
                     'a problem if you are migrating to cubicweb 3.17')
8945
ba9e3fbfa5a5 [schemaserial] serialize additional yams parameter for customs type
Vincent Michel <vincent.michel@logilab.fr>
parents: 8748
diff changeset
   225
        extra_props = {} # not yet in the schema (introduced by 3.17 migration)
9968
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   226
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   227
    # load attributes
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   228
    rql = ('Any X,SE,RT,OE,CARD,ORD,DESC,IDX,FTIDX,I18N,DFLT%(fm)s '
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   229
           'WHERE X is CWAttribute, X relation_type RT, X cardinality CARD,'
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   230
           '      X ordernum ORD, X indexed IDX, X description DESC, '
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   231
           '      X internationalizable I18N, X defaultval DFLT,%(fmsnip)s'
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   232
           '      X fulltextindexed FTIDX, X from_entity SE, X to_entity OE')
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   233
    if has_computed_attributes:
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   234
        rql = rql % {'fm': ',FM', 'fmsnip': 'X formula FM,'}
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   235
    else:
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   236
        rql = rql % {'fm': '', 'fmsnip': ''}
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   237
    for values in cnx.execute(rql, build_descr=False):
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   238
        attrs = dict(zip(
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   239
            ('rdefeid', 'seid', 'reid', 'oeid', 'cardinality',
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   240
             'order', 'description', 'indexed', 'fulltextindexed',
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   241
             'internationalizable', 'default', 'formula'), values))
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   242
        typeparams = extra_props.get(attrs['rdefeid'])
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   243
        attrs.update(json.load(typeparams) if typeparams else {})
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   244
        default = attrs['default']
9299
c5eed908117d [schema] store default attribute values in a Bytes field, allowing python objects as default values
Aurélien Campeas <aurelien.campeas@logilab.fr>
parents: 9257
diff changeset
   245
        if default is not None:
c5eed908117d [schema] store default attribute values in a Bytes field, allowing python objects as default values
Aurélien Campeas <aurelien.campeas@logilab.fr>
parents: 9257
diff changeset
   246
            if isinstance(default, Binary):
c5eed908117d [schema] store default attribute values in a Bytes field, allowing python objects as default values
Aurélien Campeas <aurelien.campeas@logilab.fr>
parents: 9257
diff changeset
   247
                # while migrating from 3.17 to 3.18, we still have to
c5eed908117d [schema] store default attribute values in a Bytes field, allowing python objects as default values
Aurélien Campeas <aurelien.campeas@logilab.fr>
parents: 9257
diff changeset
   248
                # handle String defaults
9968
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   249
                attrs['default'] = default.unzpickle()
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   250
        _add_rdef(**attrs)
50f046bf0e50 [CWEP002] properly handle serialization of computed attributes
Lea Capgen <lea.capgen@logilab.fr>
parents: 9964
diff changeset
   251
    # load relations
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   252
    for values in cnx.execute(
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   253
        'Any X,SE,RT,OE,CARD,ORD,DESC,C WHERE X is CWRelation, X relation_type RT,'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   254
        'X cardinality CARD, X ordernum ORD, X description DESC, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   255
        'X from_entity SE, X to_entity OE, X composite C', build_descr=False):
6191
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   256
        rdefeid, seid, reid, oeid, card, ord, desc, comp = values
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   257
        _add_rdef(rdefeid, seid, reid, oeid,
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   258
                  cardinality=card, description=desc, order=ord,
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   259
                  composite=comp)
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   260
    for rdef in pendingrdefs:
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   261
        try:
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   262
            rdefs = schema.add_relation_def(rdef)
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   263
        except BadSchemaDefinition:
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   264
            continue
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   265
        if rdefs is not None:
6191
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   266
            set_perms(rdefs, permsidx)
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6199
diff changeset
   267
    unique_togethers = {}
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   268
    rset = cnx.execute(
9257
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   269
    'Any X,E,R WHERE '
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   270
    'X is CWUniqueTogetherConstraint, '
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   271
    'X constraint_of E, X relations R', build_descr=False)
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   272
    for values in rset:
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   273
        uniquecstreid, eeid, releid = values
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   274
        eschema = schema.schema_by_eid(eeid)
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   275
        relations = unique_togethers.setdefault(uniquecstreid, (eschema, []))
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   276
        rel = ertidx[releid]
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   277
        if isinstance(rel, schemamod.RelationDefinitionSchema):
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   278
            # not yet migrated 3.9 database ('relations' target type changed
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   279
            # to CWRType in 3.10)
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   280
            rtype = rel.rtype.type
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   281
        else:
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   282
            rtype = str(rel)
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   283
        relations[1].append(rtype)
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   284
    for eschema, unique_together in unique_togethers.itervalues():
ce338133c92c remove cw 3.9 bw compat (bw compat other than the interface -> adapter changes)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9255
diff changeset
   285
        eschema._unique_together.append(tuple(sorted(unique_together)))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   286
    schema.infer_specialization_rules()
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   287
    cnx.commit()
9956
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   288
    schema.finalize()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   289
    schema.reading_from_database = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   291
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   292
def deserialize_ertype_permissions(cnx):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   293
    """return sect action:groups associations for the given
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   294
    entity or relation schema with its eid, according to schema's
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   295
    permissions stored in the database as [read|add|delete|update]_permission
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   296
    relations between CWEType/CWRType and CWGroup entities
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   297
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   298
    res = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   299
    for action in ('read', 'add', 'update', 'delete'):
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   300
        rql = 'Any E,N WHERE G is CWGroup, G name N, E %s_permission G' % action
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   301
        for eid, gname in cnx.execute(rql, build_descr=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   302
            res.setdefault(eid, {}).setdefault(action, []).append(gname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   303
        rql = ('Any E,X,EXPR,V WHERE X is RQLExpression, X expression EXPR, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   304
               'E %s_permission X, X mainvars V' % action)
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   305
        for eid, expreid, expr, mainvars in cnx.execute(rql, build_descr=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   306
            # we don't know yet if it's a rql expr for an entity or a relation,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   307
            # so append a tuple to differentiate from groups and so we'll be
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   308
            # able to instantiate it later
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   309
            res.setdefault(eid, {}).setdefault(action, []).append( (expr, mainvars, expreid) )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   310
    return res
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   311
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   312
def deserialize_rdef_constraints(cnx):
4760
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   313
    """return the list of relation definition's constraints as instances"""
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   314
    res = {}
9490
b3d2c4065e6a [server] use internal_cnx instead of internal_session in deserialize_schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 9489
diff changeset
   315
    for rdefeid, ceid, ct, val in cnx.execute(
4760
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   316
        'Any E, X,TN,V WHERE E constrained_by X, X is CWConstraint, '
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   317
        'X cstrtype T, T name TN, X value V', build_descr=False):
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   318
        cstr = CONSTRAINTS[ct].deserialize(val)
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   319
        cstr.eid = ceid
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   320
        res.setdefault(rdefeid, []).append(cstr)
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   321
    return res
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   322
6191
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   323
def set_perms(erschema, permsidx):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
    """set permissions on the given erschema according to the permission
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
    definition dictionary as built by deserialize_ertype_permissions for a
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
    given erschema's eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   327
    """
4570
ede247bbbf62 follow yams api change: attributes permissions are now defined for
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4475
diff changeset
   328
    # reset erschema permissions here to avoid getting yams default anyway
ede247bbbf62 follow yams api change: attributes permissions are now defined for
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4475
diff changeset
   329
    erschema.permissions = dict((action, ()) for action in erschema.ACTIONS)
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   330
    try:
6191
ece6996e6ac8 [schema deserial] fix loading of schema when mapping an entity type to an existing one
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6189
diff changeset
   331
        thispermsdict = permsidx[erschema.eid]
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   332
    except KeyError:
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   333
        return
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   334
    for action, somethings in thispermsdict.iteritems():
4570
ede247bbbf62 follow yams api change: attributes permissions are now defined for
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4475
diff changeset
   335
        erschema.permissions[action] = tuple(
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   336
            isinstance(p, tuple) and erschema.rql_expression(*p) or p
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   337
            for p in somethings)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   338
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   339
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   340
# schema / perms serialization ################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   341
9503
562cef272a35 [schemaserial] Replace 'cursor' with 'cnx'
Julien Cristau <julien.cristau@logilab.fr>
parents: 9490
diff changeset
   342
def serialize_schema(cnx, schema):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   343
    """synchronize schema and permissions in the database according to
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   344
    current schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
    """
9891
3386fd89c914 remove references to global environment variable APYCOT_ROOT
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9503
diff changeset
   346
    _title = '-> storing the schema in the database '
3386fd89c914 remove references to global environment variable APYCOT_ROOT
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9503
diff changeset
   347
    print _title,
9503
562cef272a35 [schemaserial] Replace 'cursor' with 'cnx'
Julien Cristau <julien.cristau@logilab.fr>
parents: 9490
diff changeset
   348
    execute = cnx.execute
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
    eschemas = schema.entities()
9891
3386fd89c914 remove references to global environment variable APYCOT_ROOT
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9503
diff changeset
   350
    pb_size = (len(eschemas + schema.relations())
3386fd89c914 remove references to global environment variable APYCOT_ROOT
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9503
diff changeset
   351
               + len(CONSTRAINTS)
3386fd89c914 remove references to global environment variable APYCOT_ROOT
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9503
diff changeset
   352
               + len([x for x in eschemas if x.specializes()]))
10375
28ec01db78b3 [server] Do not use progress bar when stdout is not a TTY
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 10191
diff changeset
   353
    if sys.stdout.isatty():
28ec01db78b3 [server] Do not use progress bar when stdout is not a TTY
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 10191
diff changeset
   354
        pb = ProgressBar(pb_size, title=_title)
28ec01db78b3 [server] Do not use progress bar when stdout is not a TTY
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 10191
diff changeset
   355
    else:
28ec01db78b3 [server] Do not use progress bar when stdout is not a TTY
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 10191
diff changeset
   356
        pb = DummyProgressBar()
9503
562cef272a35 [schemaserial] Replace 'cursor' with 'cnx'
Julien Cristau <julien.cristau@logilab.fr>
parents: 9490
diff changeset
   357
    groupmap = group_mapping(cnx, interactive=False)
4800
686419cd2dff nicer explanation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4799
diff changeset
   358
    # serialize all entity types, assuring CWEType is serialized first for proper
686419cd2dff nicer explanation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4799
diff changeset
   359
    # is / is_instance_of insertion
4760
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   360
    eschemas.remove(schema.eschema('CWEType'))
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   361
    eschemas.insert(0, schema.eschema('CWEType'))
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   362
    for eschema in eschemas:
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   363
        execschemarql(execute, eschema, eschema2rql(eschema, groupmap))
10115
6fd167b140cd [server] Remove useless 'is None' checks
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10095
diff changeset
   364
        pb.update()
4760
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   365
    # serialize constraint types
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   366
    cstrtypemap = {}
2597
d9c5a7e0563c [C schema seria] cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
   367
    rql = 'INSERT CWConstraintType X: X name %(ct)s'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   368
    for cstrtype in CONSTRAINTS:
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   369
        cstrtypemap[cstrtype] = execute(rql, {'ct': unicode(cstrtype)},
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   370
                                        build_descr=False)[0][0]
10115
6fd167b140cd [server] Remove useless 'is None' checks
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10095
diff changeset
   371
        pb.update()
4760
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   372
    # serialize relations
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   373
    for rschema in schema.relations():
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   374
        # skip virtual relations such as eid, has_text and identity
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   375
        if rschema in VIRTUAL_RTYPES:
10115
6fd167b140cd [server] Remove useless 'is None' checks
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10095
diff changeset
   376
            pb.update()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   377
            continue
9956
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   378
        if rschema.rule:
10553
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10411
diff changeset
   379
            execschemarql(execute, rschema, crschema2rql(rschema, groupmap))
10115
6fd167b140cd [server] Remove useless 'is None' checks
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10095
diff changeset
   380
            pb.update()
9956
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   381
            continue
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   382
        execschemarql(execute, rschema, rschema2rql(rschema, addrdef=False))
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   383
        if rschema.symmetric:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   384
            rdefs = [rdef for k, rdef in rschema.rdefs.iteritems()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   385
                     if (rdef.subject, rdef.object) == k]
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   386
        else:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   387
            rdefs = rschema.rdefs.itervalues()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   388
        for rdef in rdefs:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   389
            execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   390
                          rdef2rql(rdef, cstrtypemap, groupmap))
10115
6fd167b140cd [server] Remove useless 'is None' checks
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10095
diff changeset
   391
        pb.update()
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6199
diff changeset
   392
    # serialize unique_together constraints
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6199
diff changeset
   393
    for eschema in eschemas:
9375
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9299
diff changeset
   394
        if eschema._unique_together:
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9299
diff changeset
   395
            execschemarql(execute, eschema, uniquetogether2rqls(eschema))
6815
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6734
diff changeset
   396
    # serialize yams inheritance relationships
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   397
    for rql, kwargs in specialize2rql(schema):
4791
b9a1b16a8d26 fix schema serialization, bad repartition of changes between stable and default
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4789
diff changeset
   398
        execute(rql, kwargs, build_descr=False)
10115
6fd167b140cd [server] Remove useless 'is None' checks
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10095
diff changeset
   399
        pb.update()
9891
3386fd89c914 remove references to global environment variable APYCOT_ROOT
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9503
diff changeset
   400
    print
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   402
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   403
# high level serialization functions
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   404
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   405
def execschemarql(execute, schema, rqls):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   406
    for rql, kwargs in rqls:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   407
        kwargs['x'] = schema.eid
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   408
        rset = execute(rql, kwargs, build_descr=False)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   409
        if schema.eid is None:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   410
            schema.eid = rset[0][0]
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   411
        else:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   412
            assert rset
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   413
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   414
def erschema2rql(erschema, groupmap):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   415
    if isinstance(erschema, schemamod.EntitySchema):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   416
        return eschema2rql(erschema, groupmap=groupmap)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   417
    return rschema2rql(erschema, groupmap=groupmap)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   418
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   419
def specialize2rql(schema):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   420
    for eschema in schema.entities():
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   421
        if eschema.final:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   422
            continue
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   423
        for rql, kwargs in eschemaspecialize2rql(eschema):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   424
            yield rql, kwargs
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   425
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   426
# etype serialization
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   427
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   428
def eschema2rql(eschema, groupmap=None):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   429
    """return a list of rql insert statements to enter an entity schema
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   430
    in the database as an CWEType entity
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   431
    """
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   432
    relations, values = eschema_relations_values(eschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   433
    # NOTE: 'specializes' relation can't be inserted here since there's no
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   434
    # way to make sure the parent type is inserted before the child type
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   435
    yield 'INSERT CWEType X: %s' % ','.join(relations) , values
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   436
    # entity permissions
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   437
    if groupmap is not None:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   438
        for rql, args in _erperms2rql(eschema, groupmap):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   439
            yield rql, args
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   440
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   441
def eschema_relations_values(eschema):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   442
    values = _ervalues(eschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   443
    relations = ['X %s %%(%s)s' % (attr, attr) for attr in sorted(values)]
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   444
    return relations, values
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   445
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   446
def eschemaspecialize2rql(eschema):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   447
    specialized_type = eschema.specializes()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   448
    if specialized_type:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   449
        values = {'x': eschema.eid, 'et': specialized_type.eid}
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   450
        yield 'SET X specializes ET WHERE X eid %(x)s, ET eid %(et)s', values
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   451
9375
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9299
diff changeset
   452
def uniquetogether2rqls(eschema):
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9299
diff changeset
   453
    rql_args = []
10054
8f7291354095 [schemaserial] be robust against duplicated CWUniqueTogetherConstrain entities arising from bug in the pre 3.18 era
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9395
diff changeset
   454
    # robustness against duplicated CWUniqueTogetherConstraint (pre 3.18)
8f7291354095 [schemaserial] be robust against duplicated CWUniqueTogetherConstrain entities arising from bug in the pre 3.18 era
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9395
diff changeset
   455
    columnset = set()
9375
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9299
diff changeset
   456
    for columns in eschema._unique_together:
10054
8f7291354095 [schemaserial] be robust against duplicated CWUniqueTogetherConstrain entities arising from bug in the pre 3.18 era
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9395
diff changeset
   457
        if columns in columnset:
8f7291354095 [schemaserial] be robust against duplicated CWUniqueTogetherConstrain entities arising from bug in the pre 3.18 era
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9395
diff changeset
   458
            print ('schemaserial: skipping duplicate unique together %r %r' %
8f7291354095 [schemaserial] be robust against duplicated CWUniqueTogetherConstrain entities arising from bug in the pre 3.18 era
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9395
diff changeset
   459
                   (eschema.type, columns))
8f7291354095 [schemaserial] be robust against duplicated CWUniqueTogetherConstrain entities arising from bug in the pre 3.18 era
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9395
diff changeset
   460
            continue
8f7291354095 [schemaserial] be robust against duplicated CWUniqueTogetherConstrain entities arising from bug in the pre 3.18 era
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9395
diff changeset
   461
        columnset.add(columns)
9375
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9299
diff changeset
   462
        rql, args = _uniquetogether2rql(eschema, columns)
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9299
diff changeset
   463
        args['name'] = y2sql.unique_index_name(eschema, columns)
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9299
diff changeset
   464
        rql_args.append((rql, args))
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9299
diff changeset
   465
    return rql_args
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9299
diff changeset
   466
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9299
diff changeset
   467
def _uniquetogether2rql(eschema, unique_together):
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6199
diff changeset
   468
    relations = []
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6199
diff changeset
   469
    restrictions = []
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6199
diff changeset
   470
    substs = {}
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6199
diff changeset
   471
    for i, name in enumerate(unique_together):
6815
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6734
diff changeset
   472
        rschema = eschema.schema.rschema(name)
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6199
diff changeset
   473
        rtype = 'T%d' % i
10095
200bd6a601dc [warnings] put an end to warnings in the sqlite driver over `str` being sent instead of unicode strings
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 10074
diff changeset
   474
        substs[rtype] = unicode(rschema.type)
6815
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6734
diff changeset
   475
        relations.append('C relations %s' % rtype)
a84190d4e78c [schema] fix pb with ambiguous relation used within UniqueTogetherConstraint by having CWUniqueTogetherConstraint.relation targeting CWRType instead of CWAttribute/CWRelation. This fixes 3.10.7 migration.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6734
diff changeset
   476
        restrictions.append('%(rtype)s name %%(%(rtype)s)s' % {'rtype': rtype})
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6199
diff changeset
   477
    relations = ', '.join(relations)
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6199
diff changeset
   478
    restrictions = ', '.join(restrictions)
9375
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9299
diff changeset
   479
    rql = ('INSERT CWUniqueTogetherConstraint C: C name %%(name)s, C constraint_of X, %s '
8e88576787c3 [schema] fix unique together index handling
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9299
diff changeset
   480
           'WHERE X eid %%(x)s, %s')
6208
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6199
diff changeset
   481
    return rql % (relations, restrictions), substs
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6199
diff changeset
   482
07b176640a8c unique_together integration in CW (alf+syt)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6199
diff changeset
   483
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   484
def _ervalues(erschema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   485
    try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   486
        type_ = unicode(erschema.type)
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7815
diff changeset
   487
    except UnicodeDecodeError as e:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   488
        raise Exception("can't decode %s [was %s]" % (erschema.type, e))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   489
    try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   490
        desc = unicode(erschema.description) or u''
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 7815
diff changeset
   491
    except UnicodeDecodeError as e:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   492
        raise Exception("can't decode %s [was %s]" % (erschema.description, e))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   493
    return {
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   494
        'name': type_,
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
   495
        'final': erschema.final,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   496
        'description': desc,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   497
        }
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   498
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   499
# rtype serialization
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   500
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   501
def rschema2rql(rschema, cstrtypemap=None, addrdef=True, groupmap=None):
9956
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   502
    """generate rql insert statements to enter a relation schema
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   503
    in the database as an CWRType entity
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   504
    """
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   505
    if rschema.type == 'has_text':
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   506
        return
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   507
    relations, values = rschema_relations_values(rschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   508
    yield 'INSERT CWRType X: %s' % ','.join(relations), values
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   509
    if addrdef:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   510
        assert cstrtypemap
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   511
        # sort for testing purpose
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   512
        for rdef in sorted(rschema.rdefs.itervalues(),
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   513
                           key=lambda x: (x.subject, x.object)):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   514
            for rql, values in rdef2rql(rdef, cstrtypemap, groupmap):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   515
                yield rql, values
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   516
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   517
def rschema_relations_values(rschema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   518
    values = _ervalues(rschema)
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
   519
    values['final'] = rschema.final
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4312
diff changeset
   520
    values['symmetric'] = rschema.symmetric
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   521
    values['inlined'] = rschema.inlined
4719
aaed3f813ef8 kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4631
diff changeset
   522
    if isinstance(rschema.fulltext_container, str):
aaed3f813ef8 kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4631
diff changeset
   523
        values['fulltext_container'] = unicode(rschema.fulltext_container)
aaed3f813ef8 kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4631
diff changeset
   524
    else:
aaed3f813ef8 kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4631
diff changeset
   525
        values['fulltext_container'] = rschema.fulltext_container
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   526
    relations = ['X %s %%(%s)s' % (attr, attr) for attr in sorted(values)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   527
    return relations, values
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   528
10553
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10411
diff changeset
   529
def crschema2rql(crschema, groupmap):
9956
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   530
    relations, values = crschema_relations_values(crschema)
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   531
    yield 'INSERT CWComputedRType X: %s' % ','.join(relations), values
10553
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10411
diff changeset
   532
    if groupmap:
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10411
diff changeset
   533
        for rql, args in _erperms2rql(crschema, groupmap):
1d824df4f2bd Fix (de)serialization of ComputedRelation read permissions
Julien Cristau <julien.cristau@logilab.fr>
parents: 10411
diff changeset
   534
            yield rql, args
9956
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   535
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   536
def crschema_relations_values(crschema):
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   537
    values = _ervalues(crschema)
9963
5531f5577b50 [CWEP002 migration] support add_relation_type for computed relations
Laura Médioni <laura.medioni@logilab.fr>
parents: 9959
diff changeset
   538
    values['rule'] = unicode(crschema.rule)
9956
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   539
    # XXX why oh why?
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   540
    del values['final']
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   541
    relations = ['X %s %%(%s)s' % (attr, attr) for attr in sorted(values)]
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   542
    return relations, values
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   543
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   544
# rdef serialization
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   545
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   546
def rdef2rql(rdef, cstrtypemap, groupmap=None):
9956
19a683a0047c [CWEP002] properly handle serialization of computed relations
Lea Capgen <lea.capgen@logilab.fr>
parents: 9891
diff changeset
   547
    # don't serialize inferred relations
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   548
    if rdef.infered:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   549
        return
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   550
    relations, values = _rdef_values(rdef)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   551
    relations.append('X relation_type ER,X from_entity SE,X to_entity OE')
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   552
    values.update({'se': rdef.subject.eid, 'rt': rdef.rtype.eid, 'oe': rdef.object.eid})
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   553
    if rdef.final:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   554
        etype = 'CWAttribute'
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   555
    else:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   556
        etype = 'CWRelation'
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   557
    yield 'INSERT %s X: %s WHERE SE eid %%(se)s,ER eid %%(rt)s,OE eid %%(oe)s' % (
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   558
        etype, ','.join(relations), ), values
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   559
    for rql, values in constraints2rql(cstrtypemap, rdef.constraints):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   560
        yield rql, values
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   561
    # no groupmap means "no security insertion"
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   562
    if groupmap:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   563
        for rql, args in _erperms2rql(rdef, groupmap):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   564
            yield rql, args
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   565
10025
7b72ecc3f4d2 [migration] hackish black magic to bootstrap addition of formula attr during migration
Julien Cristau <julien.cristau@logilab.fr>
parents: 9968
diff changeset
   566
_IGNORED_PROPS = ['eid', 'constraints', 'uid', 'infered', 'permissions']
7b72ecc3f4d2 [migration] hackish black magic to bootstrap addition of formula attr during migration
Julien Cristau <julien.cristau@logilab.fr>
parents: 9968
diff changeset
   567
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   568
def _rdef_values(rdef):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   569
    amap = {'order': 'ordernum', 'default': 'defaultval'}
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   570
    values = {}
8945
ba9e3fbfa5a5 [schemaserial] serialize additional yams parameter for customs type
Vincent Michel <vincent.michel@logilab.fr>
parents: 8748
diff changeset
   571
    extra = {}
ba9e3fbfa5a5 [schemaserial] serialize additional yams parameter for customs type
Vincent Michel <vincent.michel@logilab.fr>
parents: 8748
diff changeset
   572
    for prop in rdef.rproperty_defs(rdef.object):
10025
7b72ecc3f4d2 [migration] hackish black magic to bootstrap addition of formula attr during migration
Julien Cristau <julien.cristau@logilab.fr>
parents: 9968
diff changeset
   573
        if prop in _IGNORED_PROPS:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   574
            continue
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   575
        value = getattr(rdef, prop)
8945
ba9e3fbfa5a5 [schemaserial] serialize additional yams parameter for customs type
Vincent Michel <vincent.michel@logilab.fr>
parents: 8748
diff changeset
   576
        if prop not in KNOWN_RPROPERTIES:
ba9e3fbfa5a5 [schemaserial] serialize additional yams parameter for customs type
Vincent Michel <vincent.michel@logilab.fr>
parents: 8748
diff changeset
   577
            extra[prop] = value
ba9e3fbfa5a5 [schemaserial] serialize additional yams parameter for customs type
Vincent Michel <vincent.michel@logilab.fr>
parents: 8748
diff changeset
   578
            continue
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   579
        # XXX type cast really necessary?
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   580
        if prop in ('indexed', 'fulltextindexed', 'internationalizable'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   581
            value = bool(value)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   582
        elif prop == 'ordernum':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   583
            value = int(value)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   584
        elif isinstance(value, str):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   585
            value = unicode(value)
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   586
        if value is not None and prop == 'default':
9299
c5eed908117d [schema] store default attribute values in a Bytes field, allowing python objects as default values
Aurélien Campeas <aurelien.campeas@logilab.fr>
parents: 9257
diff changeset
   587
            value = Binary.zpickle(value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   588
        values[amap.get(prop, prop)] = value
8945
ba9e3fbfa5a5 [schemaserial] serialize additional yams parameter for customs type
Vincent Michel <vincent.michel@logilab.fr>
parents: 8748
diff changeset
   589
    if extra:
ba9e3fbfa5a5 [schemaserial] serialize additional yams parameter for customs type
Vincent Michel <vincent.michel@logilab.fr>
parents: 8748
diff changeset
   590
        values['extra_props'] = Binary(json.dumps(extra))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   591
    relations = ['X %s %%(%s)s' % (attr, attr) for attr in sorted(values)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   592
    return relations, values
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   593
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   594
def constraints2rql(cstrtypemap, constraints, rdefeid=None):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   595
    for constraint in constraints:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   596
        values = {'ct': cstrtypemap[constraint.type()],
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   597
                  'value': unicode(constraint.serialize()),
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   598
                  'x': rdefeid} # when not specified, will have to be set by the caller
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   599
        yield 'INSERT CWConstraint X: X value %(value)s, X cstrtype CT, EDEF constrained_by X WHERE \
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   600
CT eid %(ct)s, EDEF eid %(x)s', values
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   601
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   602
4011
394f853bb653 [migration] write migration instructions for permissions handling on relation definition
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4010
diff changeset
   603
def _erperms2rql(erschema, groupmap):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   604
    """return rql insert statements to enter the entity or relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   605
    schema's permissions in the database as
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   606
    [read|add|delete|update]_permission relations between CWEType/CWRType
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   607
    and CWGroup entities
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   608
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   609
    for action in erschema.ACTIONS:
4631
457160bc8cfe when modifying a persistent schema, erschema may miss some permissions which wil trigger a key error, but we don't want to crash on such cases
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4570
diff changeset
   610
        try:
457160bc8cfe when modifying a persistent schema, erschema may miss some permissions which wil trigger a key error, but we don't want to crash on such cases
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4570
diff changeset
   611
            grantedto = erschema.action_permissions(action)
457160bc8cfe when modifying a persistent schema, erschema may miss some permissions which wil trigger a key error, but we don't want to crash on such cases
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4570
diff changeset
   612
        except KeyError:
457160bc8cfe when modifying a persistent schema, erschema may miss some permissions which wil trigger a key error, but we don't want to crash on such cases
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4570
diff changeset
   613
            # may occurs when modifying persistent schema
457160bc8cfe when modifying a persistent schema, erschema may miss some permissions which wil trigger a key error, but we don't want to crash on such cases
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4570
diff changeset
   614
            continue
457160bc8cfe when modifying a persistent schema, erschema may miss some permissions which wil trigger a key error, but we don't want to crash on such cases
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4570
diff changeset
   615
        for group_or_rqlexpr in grantedto:
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   616
            if isinstance(group_or_rqlexpr, basestring):
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   617
                # group
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   618
                try:
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   619
                    yield ('SET X %s_permission Y WHERE Y eid %%(g)s, X eid %%(x)s' % action,
4011
394f853bb653 [migration] write migration instructions for permissions handling on relation definition
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4010
diff changeset
   620
                           {'g': groupmap[group_or_rqlexpr]})
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   621
                except KeyError:
7219
d961184f9a86 [schema serialization] add warning on missing group
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7012
diff changeset
   622
                    print ("WARNING: group %s used in permissions for %s was ignored because it doesn't exist."
d961184f9a86 [schema serialization] add warning on missing group
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7012
diff changeset
   623
                           " You may want to add it into a precreate.py file" % (group_or_rqlexpr, erschema))
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   624
                    continue
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   625
            else:
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   626
                # rqlexpr
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   627
                rqlexpr = group_or_rqlexpr
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   628
                yield ('INSERT RQLExpression E: E expression %%(e)s, E exprtype %%(t)s, '
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   629
                       'E mainvars %%(v)s, X %s_permission E WHERE X eid %%(x)s' % action,
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   630
                       {'e': unicode(rqlexpr.expression),
7152
39c1ffc7d93f [schema, refactoring] use RQLExpression as base class for RQL constraint classes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7012
diff changeset
   631
                        'v': unicode(','.join(sorted(rqlexpr.mainvars))),
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   632
                        't': unicode(rqlexpr.__class__.__name__)})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   633
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   634
# update functions
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   635
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   636
def updateeschema2rql(eschema, eid):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   637
    relations, values = eschema_relations_values(eschema)
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   638
    values['x'] = eid
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   639
    yield 'SET %s WHERE X eid %%(x)s' % ','.join(relations), values
1630
41aadba8b29f delete-trailing-whitespaces, check table existance
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   640
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   641
def updaterschema2rql(rschema, eid):
9964
f4a3ee05cf9d [CWEP002 migration] support sync_schema_props_perms for computed relations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9963
diff changeset
   642
    if rschema.rule:
f4a3ee05cf9d [CWEP002 migration] support sync_schema_props_perms for computed relations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9963
diff changeset
   643
        yield ('SET X rule %(r)s WHERE X eid %(x)s',
f4a3ee05cf9d [CWEP002 migration] support sync_schema_props_perms for computed relations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9963
diff changeset
   644
               {'x': eid, 'r': unicode(rschema.rule)})
f4a3ee05cf9d [CWEP002 migration] support sync_schema_props_perms for computed relations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9963
diff changeset
   645
    else:
f4a3ee05cf9d [CWEP002 migration] support sync_schema_props_perms for computed relations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9963
diff changeset
   646
        relations, values = rschema_relations_values(rschema)
f4a3ee05cf9d [CWEP002 migration] support sync_schema_props_perms for computed relations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9963
diff changeset
   647
        values['x'] = eid
f4a3ee05cf9d [CWEP002 migration] support sync_schema_props_perms for computed relations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9963
diff changeset
   648
        yield 'SET %s WHERE X eid %%(x)s' % ','.join(relations), values
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   649
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   650
def updaterdef2rql(rdef, eid):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   651
    relations, values = _rdef_values(rdef)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   652
    values['x'] = eid
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   653
    yield 'SET %s WHERE X eid %%(x)s' % ','.join(relations), values