server/schemaserial.py
author Aurelien Campeas <aurelien.campeas@logilab.fr>
Thu, 15 Apr 2010 14:07:25 +0200
branchstable
changeset 5277 92b827b3830e
parent 4835 13b0b96d7982
child 5379 c082dea0731b
child 5421 8167de96c523
permissions -rw-r--r--
[doc] save idiotic changes to some drawing
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     1
"""functions for schema / permissions (de)serialization using RQL
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     2
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     3
:organization: Logilab
4212
ab6573088b4a update copyright: welcome 2010
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3897
diff changeset
     4
:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     5
:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr
1977
606923dff11b big bunch of copyright / docstring update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1913
diff changeset
     6
:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     7
"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     8
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     9
3854
8633cd05b6b5 no progress bar in apycot environment
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    10
import os
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    11
from itertools import chain
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    12
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    13
from logilab.common.shellutils import ProgressBar
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    14
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    15
from yams import schema as schemamod, buildobjs as ybo
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    16
2596
d02eed70937f [R repo, schema] use VIRTUAL_RTYPES const
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2458
diff changeset
    17
from cubicweb.schema import CONSTRAINTS, ETYPE_NAME_MAP, VIRTUAL_RTYPES
1869
642a1a120a92 should use sql prefix here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1630
diff changeset
    18
from cubicweb.server import sqlutils
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
def group_mapping(cursor, interactive=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
    """create a group mapping from an rql cursor
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
    A group mapping has standard group names as key (managers, owners at least)
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
    24
    and the actual CWGroup entity's eid as associated value.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
    In interactive mode (the default), missing groups'eid will be prompted
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
    from the user.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    28
    res = {}
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    29
    for eid, name in cursor.execute('Any G, N WHERE G is CWGroup, G name N',
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    30
                                    build_descr=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    31
        res[name] = eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    32
    if not interactive:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    33
        return res
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    34
    missing = [g for g in ('owners', 'managers', 'users', 'guests') if not g in res]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    35
    if missing:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    36
        print 'some native groups are missing but the following groups have been found:'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    37
        print '\n'.join('* %s (%s)' % (n, eid) for n, eid in res.items())
1630
41aadba8b29f delete-trailing-whitespaces, check table existance
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
    38
        print
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    39
        print 'enter the eid of a to group to map to each missing native group'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    40
        print 'or just type enter to skip permissions granted to a group'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    41
        for group in missing:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    42
            while True:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    43
                value = raw_input('eid for group %s: ' % group).strip()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    44
                if not value:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    45
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    46
                try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    47
                    res[group] = int(value)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    48
                except ValueError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    49
                    print 'eid should be an integer'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    51
    return res
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
    53
def cstrtype_mapping(cursor):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
    54
    """cached constraint types mapping"""
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
    55
    return dict(cursor.execute('Any T, X WHERE X is CWConstraintType, X name T'))
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
    56
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    57
# schema / perms deserialization ##############################################
4791
b9a1b16a8d26 fix schema serialization, bad repartition of changes between stable and default
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4789
diff changeset
    58
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    59
def deserialize_schema(schema, session):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    60
    """return a schema according to information stored in an rql database
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
    61
    as CWRType and CWEType entities
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    62
    """
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
    63
    repo = session.repo
2044
d5589c5bc4e6 alter table in schema serial when a renamed type is detected
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    64
    dbhelper = repo.system_source.dbhelper
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4312
diff changeset
    65
    # 3.6 migration
4475
37c413a07216 kill most pre 3.2 bw compat code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4467
diff changeset
    66
    sqlcu = session.pool['system']
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4312
diff changeset
    67
    sqlcu.execute("SELECT * FROM cw_CWRType WHERE cw_name='symetric'")
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4312
diff changeset
    68
    if sqlcu.fetchall():
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4312
diff changeset
    69
        sql = dbhelper.sql_rename_col('cw_CWRType', 'cw_symetric', 'cw_symmetric',
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4312
diff changeset
    70
                                      dbhelper.TYPE_MAPPING['Boolean'], True)
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4312
diff changeset
    71
        sqlcu.execute(sql)
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4312
diff changeset
    72
        sqlcu.execute("UPDATE cw_CWRType SET cw_name='symmetric' WHERE cw_name='symetric'")
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    73
    sidx = {}
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    74
    permsdict = deserialize_ertype_permissions(session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    75
    schema.reading_from_database = True
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    76
    for eid, etype, desc in session.execute(
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    77
        'Any X, N, D WHERE X is CWEType, X name N, X description D',
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    78
        build_descr=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    79
        # base types are already in the schema, skip them
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    80
        if etype in schemamod.BASE_TYPES:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    81
            # just set the eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    82
            eschema = schema.eschema(etype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    83
            eschema.eid = eid
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
    84
            sidx[eid] = eschema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    85
            continue
2044
d5589c5bc4e6 alter table in schema serial when a renamed type is detected
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    86
        if etype in ETYPE_NAME_MAP:
d5589c5bc4e6 alter table in schema serial when a renamed type is detected
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    87
            netype = ETYPE_NAME_MAP[etype]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    88
            # can't use write rql queries at this point, use raw sql
1869
642a1a120a92 should use sql prefix here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1630
diff changeset
    89
            session.system_sql('UPDATE %(p)sCWEType SET %(p)sname=%%(n)s WHERE %(p)seid=%%(x)s'
642a1a120a92 should use sql prefix here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1630
diff changeset
    90
                               % {'p': sqlutils.SQL_PREFIX},
2044
d5589c5bc4e6 alter table in schema serial when a renamed type is detected
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    91
                               {'x': eid, 'n': netype})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
            session.system_sql('UPDATE entities SET type=%(n)s WHERE type=%(x)s',
2044
d5589c5bc4e6 alter table in schema serial when a renamed type is detected
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    93
                               {'x': etype, 'n': netype})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    94
            session.commit(False)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
                session.system_sql('UPDATE deleted_entities SET type=%(n)s WHERE type=%(x)s',
2044
d5589c5bc4e6 alter table in schema serial when a renamed type is detected
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    97
                                   {'x': etype, 'n': netype})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    98
            except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    99
                pass
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   100
            tocleanup = [eid]
1630
41aadba8b29f delete-trailing-whitespaces, check table existance
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   101
            tocleanup += (eid for eid, (eidetype, uri, extid) in repo._type_source_cache.items()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   102
                          if etype == eidetype)
1630
41aadba8b29f delete-trailing-whitespaces, check table existance
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   103
            repo.clear_caches(tocleanup)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   104
            session.commit(False)
2044
d5589c5bc4e6 alter table in schema serial when a renamed type is detected
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   105
            etype = netype
2126
a25859917ccc stop using meta attribute from yams schema. Use instead sets defining meta relations and another defining schema types. Refactor various schema view based on this
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2044
diff changeset
   106
        etype = ybo.EntityType(name=etype, description=desc, eid=eid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   107
        eschema = schema.add_entity_type(etype)
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   108
        sidx[eid] = eschema
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   109
        set_perms(eschema, permsdict)
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   110
    for etype, stype in session.execute(
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   111
        'Any XN, ETN WHERE X is CWEType, X name XN, X specializes ET, ET name ETN',
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   112
        build_descr=False):
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   113
        schema.eschema(etype)._specialized_type = stype
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   114
        schema.eschema(stype)._specialized_by.append(etype)
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   115
    for eid, rtype, desc, sym, il, ftc in session.execute(
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   116
        'Any X,N,D,S,I,FTC WHERE X is CWRType, X name N, X description D, '
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4312
diff changeset
   117
        'X symmetric S, X inlined I, X fulltext_container FTC', build_descr=False):
2126
a25859917ccc stop using meta attribute from yams schema. Use instead sets defining meta relations and another defining schema types. Refactor various schema view based on this
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2044
diff changeset
   118
        rtype = ybo.RelationType(name=rtype, description=desc,
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4312
diff changeset
   119
                                 symmetric=bool(sym), inlined=bool(il),
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   120
                                 fulltext_container=ftc, eid=eid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   121
        rschema = schema.add_relation_type(rtype)
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   122
        sidx[eid] = rschema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   123
    cstrsdict = deserialize_rdef_constraints(session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   124
    for values in session.execute(
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   125
        'Any X,SE,RT,OE,CARD,ORD,DESC,IDX,FTIDX,I18N,DFLT WHERE X is CWAttribute,'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   126
        'X relation_type RT, X cardinality CARD, X ordernum ORD, X indexed IDX,'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   127
        'X description DESC, X internationalizable I18N, X defaultval DFLT,'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   128
        'X fulltextindexed FTIDX, X from_entity SE, X to_entity OE',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
        build_descr=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   130
        rdefeid, seid, reid, teid, card, ord, desc, idx, ftidx, i18n, default = values
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   131
        rdef = ybo.RelationDefinition(sidx[seid].type, sidx[reid].type, sidx[teid].type,
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   132
                                      cardinality=card,
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   133
                                      constraints=cstrsdict.get(rdefeid, ()),
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   134
                                      order=ord, description=desc,
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   135
                                      indexed=idx, fulltextindexed=ftidx,
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   136
                                      internationalizable=i18n,
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   137
                                      default=default, eid=rdefeid)
4003
b9436fe77c9e fix bad merge
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 3998
diff changeset
   138
        rdefs = schema.add_relation_def(rdef)
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4312
diff changeset
   139
        # rdefs can be None on duplicated relation definitions (e.g. symmetrics)
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   140
        if rdefs is not None:
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   141
            set_perms(rdefs, permsdict)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
    for values in session.execute(
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   143
        'Any X,SE,RT,OE,CARD,ORD,DESC,C WHERE X is CWRelation, X relation_type RT,'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   144
        'X cardinality CARD, X ordernum ORD, X description DESC, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   145
        'X from_entity SE, X to_entity OE, X composite C', build_descr=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   146
        rdefeid, seid, reid, teid, card, ord, desc, c = values
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   147
        rdef = ybo.RelationDefinition(sidx[seid].type, sidx[reid].type, sidx[teid].type,
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   148
                                      constraints=cstrsdict.get(rdefeid, ()),
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   149
                                      cardinality=card, order=ord, description=desc,
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   150
                                      composite=c,  eid=rdefeid)
4003
b9436fe77c9e fix bad merge
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 3998
diff changeset
   151
        rdefs = schema.add_relation_def(rdef)
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4312
diff changeset
   152
        # rdefs can be None on duplicated relation definitions (e.g. symmetrics)
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   153
        if rdefs is not None:
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   154
            set_perms(rdefs, permsdict)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   155
    schema.infer_specialization_rules()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   156
    session.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   157
    schema.reading_from_database = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   158
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   159
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
def deserialize_ertype_permissions(session):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   161
    """return sect action:groups associations for the given
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   162
    entity or relation schema with its eid, according to schema's
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   163
    permissions stored in the database as [read|add|delete|update]_permission
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   164
    relations between CWEType/CWRType and CWGroup entities
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   165
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   166
    res = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   167
    for action in ('read', 'add', 'update', 'delete'):
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   168
        rql = 'Any E,N WHERE G is CWGroup, G name N, E %s_permission G' % action
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   169
        for eid, gname in session.execute(rql, build_descr=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   170
            res.setdefault(eid, {}).setdefault(action, []).append(gname)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   171
        rql = ('Any E,X,EXPR,V WHERE X is RQLExpression, X expression EXPR, '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   172
               'E %s_permission X, X mainvars V' % action)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   173
        for eid, expreid, expr, mainvars in session.execute(rql, build_descr=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   174
            # we don't know yet if it's a rql expr for an entity or a relation,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   175
            # so append a tuple to differentiate from groups and so we'll be
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   176
            # able to instantiate it later
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   177
            res.setdefault(eid, {}).setdefault(action, []).append( (expr, mainvars, expreid) )
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   178
    return res
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   179
4760
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   180
def deserialize_rdef_constraints(session):
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   181
    """return the list of relation definition's constraints as instances"""
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   182
    res = {}
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   183
    for rdefeid, ceid, ct, val in session.execute(
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   184
        'Any E, X,TN,V WHERE E constrained_by X, X is CWConstraint, '
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   185
        'X cstrtype T, T name TN, X value V', build_descr=False):
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   186
        cstr = CONSTRAINTS[ct].deserialize(val)
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   187
        cstr.eid = ceid
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   188
        res.setdefault(rdefeid, []).append(cstr)
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   189
    return res
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   190
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   191
def set_perms(erschema, permsdict):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   192
    """set permissions on the given erschema according to the permission
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   193
    definition dictionary as built by deserialize_ertype_permissions for a
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   194
    given erschema's eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   195
    """
4570
ede247bbbf62 follow yams api change: attributes permissions are now defined for
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4475
diff changeset
   196
    # reset erschema permissions here to avoid getting yams default anyway
ede247bbbf62 follow yams api change: attributes permissions are now defined for
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4475
diff changeset
   197
    erschema.permissions = dict((action, ()) for action in erschema.ACTIONS)
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   198
    try:
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   199
        thispermsdict = permsdict[erschema.eid]
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   200
    except KeyError:
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   201
        return
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   202
    for action, somethings in thispermsdict.iteritems():
4570
ede247bbbf62 follow yams api change: attributes permissions are now defined for
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4475
diff changeset
   203
        # XXX cw < 3.6.1 bw compat
ede247bbbf62 follow yams api change: attributes permissions are now defined for
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4475
diff changeset
   204
        if isinstance(erschema, schemamod.RelationDefinitionSchema) and erschema.final and action == 'add':
ede247bbbf62 follow yams api change: attributes permissions are now defined for
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4475
diff changeset
   205
            action = 'update'
ede247bbbf62 follow yams api change: attributes permissions are now defined for
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4475
diff changeset
   206
        erschema.permissions[action] = tuple(
4312
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   207
            isinstance(p, tuple) and erschema.rql_expression(*p) or p
740397c24ec3 minor schema deserialization optimizations
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   208
            for p in somethings)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   209
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   210
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   211
# schema / perms serialization ################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   212
4760
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   213
def serialize_schema(cursor, schema):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   214
    """synchronize schema and permissions in the database according to
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   215
    current schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   216
    """
3737
e8c669e33689 finally
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3732
diff changeset
   217
    quiet = os.environ.get('APYCOT_ROOT')
3723
a961c7cb2644 be quiet w/ apycot
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   218
    if not quiet:
a961c7cb2644 be quiet w/ apycot
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   219
        _title = '-> storing the schema in the database '
a961c7cb2644 be quiet w/ apycot
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   220
        print _title,
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4808
diff changeset
   221
    execute = cursor.execute
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   222
    eschemas = schema.entities()
4760
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   223
    if not quiet:
4791
b9a1b16a8d26 fix schema serialization, bad repartition of changes between stable and default
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4789
diff changeset
   224
        pb_size = (len(eschemas + schema.relations())
b9a1b16a8d26 fix schema serialization, bad repartition of changes between stable and default
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4789
diff changeset
   225
                   + len(CONSTRAINTS)
b9a1b16a8d26 fix schema serialization, bad repartition of changes between stable and default
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4789
diff changeset
   226
                   + len([x for x in eschemas if x.specializes()]))
2396
8bfb99d7bbcc [cw-ctl] improve dialog messages
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2395
diff changeset
   227
        pb = ProgressBar(pb_size, title=_title)
3857
6676a839dc97 ensure pb is not None
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3854
diff changeset
   228
    else:
6676a839dc97 ensure pb is not None
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3854
diff changeset
   229
        pb = None
4800
686419cd2dff nicer explanation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4799
diff changeset
   230
    groupmap = group_mapping(cursor, interactive=False)
686419cd2dff nicer explanation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4799
diff changeset
   231
    # serialize all entity types, assuring CWEType is serialized first for proper
686419cd2dff nicer explanation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4799
diff changeset
   232
    # is / is_instance_of insertion
4760
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   233
    eschemas.remove(schema.eschema('CWEType'))
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   234
    eschemas.insert(0, schema.eschema('CWEType'))
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   235
    for eschema in eschemas:
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   236
        execschemarql(execute, eschema, eschema2rql(eschema, groupmap))
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   237
        if pb is not None:
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   238
            pb.update()
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   239
    # serialize constraint types
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   240
    cstrtypemap = {}
2597
d9c5a7e0563c [C schema seria] cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
   241
    rql = 'INSERT CWConstraintType X: X name %(ct)s'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   242
    for cstrtype in CONSTRAINTS:
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   243
        cstrtypemap[cstrtype] = execute(rql, {'ct': unicode(cstrtype)},
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   244
                                        build_descr=False)[0][0]
3857
6676a839dc97 ensure pb is not None
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3854
diff changeset
   245
        if pb is not None:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   246
            pb.update()
4760
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   247
    # serialize relations
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   248
    for rschema in schema.relations():
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   249
        # skip virtual relations such as eid, has_text and identity
fe0e307b9b70 minor schema serialization optimization:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4722
diff changeset
   250
        if rschema in VIRTUAL_RTYPES:
3857
6676a839dc97 ensure pb is not None
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3854
diff changeset
   251
            if pb is not None:
6676a839dc97 ensure pb is not None
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3854
diff changeset
   252
                pb.update()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   253
            continue
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   254
        execschemarql(execute, rschema, rschema2rql(rschema, addrdef=False))
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   255
        if rschema.symmetric:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   256
            rdefs = [rdef for k, rdef in rschema.rdefs.iteritems()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   257
                     if (rdef.subject, rdef.object) == k]
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   258
        else:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   259
            rdefs = rschema.rdefs.itervalues()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   260
        for rdef in rdefs:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   261
            execschemarql(execute, rdef,
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   262
                          rdef2rql(rdef, cstrtypemap, groupmap))
3857
6676a839dc97 ensure pb is not None
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3854
diff changeset
   263
        if pb is not None:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   264
            pb.update()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   265
    for rql, kwargs in specialize2rql(schema):
4791
b9a1b16a8d26 fix schema serialization, bad repartition of changes between stable and default
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4789
diff changeset
   266
        execute(rql, kwargs, build_descr=False)
3857
6676a839dc97 ensure pb is not None
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3854
diff changeset
   267
        if pb is not None:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   268
            pb.update()
3723
a961c7cb2644 be quiet w/ apycot
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   269
    if not quiet:
a961c7cb2644 be quiet w/ apycot
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   270
        print
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   271
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   272
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   273
# high level serialization functions
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   274
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   275
def execschemarql(execute, schema, rqls):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   276
    for rql, kwargs in rqls:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   277
        kwargs['x'] = schema.eid
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   278
        rset = execute(rql, kwargs, build_descr=False)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   279
        if schema.eid is None:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   280
            schema.eid = rset[0][0]
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   281
        else:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   282
            assert rset
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   283
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   284
def erschema2rql(erschema, groupmap):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   285
    if isinstance(erschema, schemamod.EntitySchema):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   286
        return eschema2rql(erschema, groupmap=groupmap)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   287
    return rschema2rql(erschema, groupmap=groupmap)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   288
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   289
def specialize2rql(schema):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   290
    for eschema in schema.entities():
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   291
        if eschema.final:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   292
            continue
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   293
        for rql, kwargs in eschemaspecialize2rql(eschema):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   294
            yield rql, kwargs
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   295
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   296
# etype serialization
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   297
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   298
def eschema2rql(eschema, groupmap=None):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   299
    """return a list of rql insert statements to enter an entity schema
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   300
    in the database as an CWEType entity
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   301
    """
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   302
    relations, values = eschema_relations_values(eschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   303
    # NOTE: 'specializes' relation can't be inserted here since there's no
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   304
    # way to make sure the parent type is inserted before the child type
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   305
    yield 'INSERT CWEType X: %s' % ','.join(relations) , values
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   306
    # entity permissions
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   307
    if groupmap is not None:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   308
        for rql, args in _erperms2rql(eschema, groupmap):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   309
            yield rql, args
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   310
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   311
def eschema_relations_values(eschema):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   312
    values = _ervalues(eschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   313
    relations = ['X %s %%(%s)s' % (attr, attr) for attr in sorted(values)]
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   314
    return relations, values
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   315
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   316
def eschemaspecialize2rql(eschema):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   317
    specialized_type = eschema.specializes()
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   318
    if specialized_type:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   319
        values = {'x': eschema.eid, 'et': specialized_type.eid}
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   320
        yield 'SET X specializes ET WHERE X eid %(x)s, ET eid %(et)s', values
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   321
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   322
def _ervalues(erschema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
    try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
        type_ = unicode(erschema.type)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
    except UnicodeDecodeError, e:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
        raise Exception("can't decode %s [was %s]" % (erschema.type, e))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   327
    try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
        desc = unicode(erschema.description) or u''
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
    except UnicodeDecodeError, e:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   330
        raise Exception("can't decode %s [was %s]" % (erschema.description, e))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
    return {
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   332
        'name': type_,
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
   333
        'final': erschema.final,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   334
        'description': desc,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   335
        }
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   336
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   337
# rtype serialization
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   338
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   339
def rschema2rql(rschema, cstrtypemap=None, addrdef=True, groupmap=None):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   340
    """return a list of rql insert statements to enter a relation schema
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   341
    in the database as an CWRType entity
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   342
    """
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   343
    if rschema.type == 'has_text':
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   344
        return
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   345
    relations, values = rschema_relations_values(rschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   346
    yield 'INSERT CWRType X: %s' % ','.join(relations), values
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   347
    if addrdef:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   348
        assert cstrtypemap
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   349
        # sort for testing purpose
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   350
        for rdef in sorted(rschema.rdefs.itervalues(),
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   351
                           key=lambda x: (x.subject, x.object)):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   352
            for rql, values in rdef2rql(rdef, cstrtypemap, groupmap):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   353
                yield rql, values
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   354
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   355
def rschema_relations_values(rschema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   356
    values = _ervalues(rschema)
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2955
diff changeset
   357
    values['final'] = rschema.final
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4312
diff changeset
   358
    values['symmetric'] = rschema.symmetric
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   359
    values['inlined'] = rschema.inlined
4719
aaed3f813ef8 kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4631
diff changeset
   360
    if isinstance(rschema.fulltext_container, str):
aaed3f813ef8 kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4631
diff changeset
   361
        values['fulltext_container'] = unicode(rschema.fulltext_container)
aaed3f813ef8 kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4631
diff changeset
   362
    else:
aaed3f813ef8 kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4631
diff changeset
   363
        values['fulltext_container'] = rschema.fulltext_container
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   364
    relations = ['X %s %%(%s)s' % (attr, attr) for attr in sorted(values)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   365
    return relations, values
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   366
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   367
# rdef serialization
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   368
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   369
def rdef2rql(rdef, cstrtypemap, groupmap=None):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   370
    # don't serialize infered relations
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   371
    if rdef.infered:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   372
        return
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   373
    relations, values = _rdef_values(rdef)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   374
    relations.append('X relation_type ER,X from_entity SE,X to_entity OE')
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   375
    values.update({'se': rdef.subject.eid, 'rt': rdef.rtype.eid, 'oe': rdef.object.eid})
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   376
    if rdef.final:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   377
        etype = 'CWAttribute'
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   378
    else:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   379
        etype = 'CWRelation'
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   380
    yield 'INSERT %s X: %s WHERE SE eid %%(se)s,ER eid %%(rt)s,OE eid %%(oe)s' % (
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   381
        etype, ','.join(relations), ), values
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   382
    for rql, values in constraints2rql(cstrtypemap, rdef.constraints):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   383
        yield rql, values
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   384
    # no groupmap means "no security insertion"
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   385
    if groupmap:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   386
        for rql, args in _erperms2rql(rdef, groupmap):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   387
            yield rql, args
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   388
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   389
def _rdef_values(rdef):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   390
    amap = {'order': 'ordernum', 'default': 'defaultval'}
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   391
    values = {}
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   392
    for prop, default in rdef.rproperty_defs(rdef.object).iteritems():
4003
b9436fe77c9e fix bad merge
Sandrine Ribeau <sandrine.ribeau@logilab.fr>
parents: 3998
diff changeset
   393
        if prop in ('eid', 'constraints', 'uid', 'infered', 'permissions'):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   394
            continue
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   395
        value = getattr(rdef, prop)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   396
        # XXX type cast really necessary?
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   397
        if prop in ('indexed', 'fulltextindexed', 'internationalizable'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   398
            value = bool(value)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   399
        elif prop == 'ordernum':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   400
            value = int(value)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
        elif isinstance(value, str):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   402
            value = unicode(value)
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   403
        if value is not None and prop == 'default':
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   404
            if value is False:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   405
                value = u''
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   406
            if not isinstance(value, unicode):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   407
                value = unicode(value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   408
        values[amap.get(prop, prop)] = value
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   409
    relations = ['X %s %%(%s)s' % (attr, attr) for attr in sorted(values)]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   410
    return relations, values
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   411
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   412
def constraints2rql(cstrtypemap, constraints, rdefeid=None):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   413
    for constraint in constraints:
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   414
        values = {'ct': cstrtypemap[constraint.type()],
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   415
                  'value': unicode(constraint.serialize()),
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   416
                  'x': rdefeid} # when not specified, will have to be set by the caller
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   417
        yield 'INSERT CWConstraint X: X value %(value)s, X cstrtype CT, EDEF constrained_by X WHERE \
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   418
CT eid %(ct)s, EDEF eid %(x)s', values
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   419
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   420
4011
394f853bb653 [migration] write migration instructions for permissions handling on relation definition
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4010
diff changeset
   421
def _erperms2rql(erschema, groupmap):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   422
    """return rql insert statements to enter the entity or relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
    schema's permissions in the database as
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   424
    [read|add|delete|update]_permission relations between CWEType/CWRType
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   425
    and CWGroup entities
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   426
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   427
    for action in erschema.ACTIONS:
4631
457160bc8cfe when modifying a persistent schema, erschema may miss some permissions which wil trigger a key error, but we don't want to crash on such cases
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4570
diff changeset
   428
        try:
457160bc8cfe when modifying a persistent schema, erschema may miss some permissions which wil trigger a key error, but we don't want to crash on such cases
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4570
diff changeset
   429
            grantedto = erschema.action_permissions(action)
457160bc8cfe when modifying a persistent schema, erschema may miss some permissions which wil trigger a key error, but we don't want to crash on such cases
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4570
diff changeset
   430
        except KeyError:
457160bc8cfe when modifying a persistent schema, erschema may miss some permissions which wil trigger a key error, but we don't want to crash on such cases
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4570
diff changeset
   431
            # may occurs when modifying persistent schema
457160bc8cfe when modifying a persistent schema, erschema may miss some permissions which wil trigger a key error, but we don't want to crash on such cases
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4570
diff changeset
   432
            continue
457160bc8cfe when modifying a persistent schema, erschema may miss some permissions which wil trigger a key error, but we don't want to crash on such cases
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4570
diff changeset
   433
        for group_or_rqlexpr in grantedto:
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   434
            if isinstance(group_or_rqlexpr, basestring):
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   435
                # group
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   436
                try:
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   437
                    yield ('SET X %s_permission Y WHERE Y eid %%(g)s, X eid %%(x)s' % action,
4011
394f853bb653 [migration] write migration instructions for permissions handling on relation definition
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4010
diff changeset
   438
                           {'g': groupmap[group_or_rqlexpr]})
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   439
                except KeyError:
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   440
                    continue
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   441
            else:
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   442
                # rqlexpr
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   443
                rqlexpr = group_or_rqlexpr
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   444
                yield ('INSERT RQLExpression E: E expression %%(e)s, E exprtype %%(t)s, '
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   445
                       'E mainvars %%(v)s, X %s_permission E WHERE X eid %%(x)s' % action,
3877
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   446
                       {'e': unicode(rqlexpr.expression),
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   447
                        'v': unicode(rqlexpr.mainvars),
7ca53fc72a0a reldefsecurity branch :
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3876
diff changeset
   448
                        't': unicode(rqlexpr.__class__.__name__)})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   449
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   450
# update functions
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   451
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   452
def updateeschema2rql(eschema, eid):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   453
    relations, values = eschema_relations_values(eschema)
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   454
    values['x'] = eid
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   455
    yield 'SET %s WHERE X eid %%(x)s' % ','.join(relations), values
1630
41aadba8b29f delete-trailing-whitespaces, check table existance
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   456
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   457
def updaterschema2rql(rschema, eid):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   458
    relations, values = rschema_relations_values(rschema)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   459
    values['x'] = eid
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   460
    yield 'SET %s WHERE X eid %%(x)s' % ','.join(relations), values
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   461
4763
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   462
def updaterdef2rql(rdef, eid):
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   463
    relations, values = _rdef_values(rdef)
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   464
    values['x'] = eid
81b0df087375 schema serialization optimization by using eids instead of type names. Heavy refactoring/cleanup on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4760
diff changeset
   465
    yield 'SET %s WHERE X eid %%(x)s' % ','.join(relations), values