cubicweb/server/schema2sql.py
author Denis Laxalde <denis.laxalde@logilab.fr>
Tue, 12 Dec 2017 11:17:25 +0100
changeset 12245 cd760c411242
parent 11811 f09efeead7f9
child 12355 c703dc95c82e
permissions -rw-r--r--
[pyramid] Let logging be configured through .ini file We do not initialize logging in CubicWebPyramidConfiguration thus preventing logging to be started with cubicweb's configuration. On the other hand, any logging configuration defined in ``development.ini`` file will be loaded by whatever start the instance using this file (i.e. pserve, gunicorn, etc.). The benefit is that logging can now be configured per "qualified name" in this file (i.e. one can easily set the DEBUG level for the application cube while keeping all dependencies' level to WARNING). In the development.ini template, we add logger configurations for "logilab" and "cubicweb" qualified names (along with those of the application cube).
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
11287
b537c07e3bdc pep8 schema2sql
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11005
diff changeset
     1
# copyright 2004-2016 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     3
#
10481
6ac4b1726e9f [schema2sql] properly consider skip_relations parameters.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10444
diff changeset
     4
# This file is part of cubicweb.
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     5
#
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     6
# yams is free software: you can redistribute it and/or modify it under the
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
     9
# any later version.
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    10
#
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    11
# yams is distributed in the hope that it will be useful, but WITHOUT ANY
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    12
# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    13
# A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    14
# details.
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    15
#
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    17
# with yams. If not, see <http://www.gnu.org/licenses/>.
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    18
"""write a schema as sql"""
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    19
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    20
from hashlib import md5
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    21
11360
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    22
from six import string_types, text_type
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    23
from six.moves import range
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    24
10444
fb7c1013189e [schema2sql] support NOW and TODAY in check constraints
Julien Cristau <julien.cristau@logilab.fr>
parents: 10443
diff changeset
    25
from yams.constraints import (SizeConstraint, UniqueConstraint, Attribute,
fb7c1013189e [schema2sql] support NOW and TODAY in check constraints
Julien Cristau <julien.cristau@logilab.fr>
parents: 10443
diff changeset
    26
                              NOW, TODAY)
11359
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
    27
from logilab import database
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
    28
from logilab.common.decorators import monkeypatch
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    29
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    30
# default are usually not handled at the sql level. If you want them, set
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    31
# SET_DEFAULT to True
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    32
SET_DEFAULT = False
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    33
11287
b537c07e3bdc pep8 schema2sql
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11005
diff changeset
    34
11359
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
    35
# backport fix for lgdb #6662663
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
    36
@monkeypatch(database._GenericAdvFuncHelper)
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
    37
def sql_create_index(self, table, column, unique=False):
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
    38
    idx = self._index_name(table, column, unique)
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
    39
    if unique:
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
    40
        return 'ALTER TABLE %s ADD CONSTRAINT %s UNIQUE(%s)' % (table, idx, column)
11359
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
    41
    else:
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
    42
        return 'CREATE INDEX %s ON %s(%s)' % (idx, table, column)
11359
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
    43
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
    44
11360
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    45
@monkeypatch(database._GenericAdvFuncHelper)
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    46
def _index_name(self, table, column, unique=False):
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    47
    if unique:
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    48
        return build_index_name(table, [column], prefix='key_')
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    49
    else:
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    50
        return build_index_name(table, [column], prefix='idx_')
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    51
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    52
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    53
def build_index_name(table, columns, prefix='idx_'):
11408
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
    54
    """Return a predictable-but-size-constrained name for an index on `table(*columns)`, using an
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
    55
    md5 hash.
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
    56
    """
11360
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    57
    return '%s%s' % (prefix, md5((table +
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    58
                                  ',' +
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    59
                                  ','.join(sorted(columns))).encode('ascii')).hexdigest())
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    60
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    61
10481
6ac4b1726e9f [schema2sql] properly consider skip_relations parameters.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10444
diff changeset
    62
def rschema_has_table(rschema, skip_relations):
11408
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
    63
    """Return True if the given schema should have a table in the database."""
10481
6ac4b1726e9f [schema2sql] properly consider skip_relations parameters.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10444
diff changeset
    64
    return not (rschema.final or rschema.inlined or rschema.rule or rschema.type in skip_relations)
6ac4b1726e9f [schema2sql] properly consider skip_relations parameters.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10444
diff changeset
    65
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    66
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    67
def schema2sql(dbhelper, schema, skip_entities=(), skip_relations=(), prefix=''):
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
    68
    """Yield SQL statements to create a database schema for the given Yams schema.
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
    69
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
    70
    `prefix` may be a string that will be prepended to all table / column names (usually, 'cw_').
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    71
    """
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    72
    for etype in sorted(schema.entities()):
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    73
        eschema = schema.eschema(etype)
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    74
        if eschema.final or eschema.type in skip_entities:
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    75
            continue
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
    76
        for sql in eschema2sql(dbhelper, eschema, skip_relations, prefix):
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
    77
            yield sql
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    78
    for rtype in sorted(schema.relations()):
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    79
        rschema = schema.rschema(rtype)
10481
6ac4b1726e9f [schema2sql] properly consider skip_relations parameters.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10444
diff changeset
    80
        if rschema_has_table(rschema, skip_relations):
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
    81
            for sql in rschema2sql(rschema):
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
    82
                yield sql
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    83
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    84
11408
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
    85
def unique_index_name(eschema, attrs):
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
    86
    """Return a predictable-but-size-constrained name for a multi-columns unique index on
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
    87
    given attributes of the entity schema (actually, the later may be a schema or a string).
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
    88
    """
11362
ebe75d73acdd [schema sync] Rename index when an entity type is renamed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11360
diff changeset
    89
    # keep giving eschema instead of table name for bw compat
ebe75d73acdd [schema sync] Rename index when an entity type is renamed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11360
diff changeset
    90
    table = text_type(eschema)
11360
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
    91
    # unique_index_name is used as name of CWUniqueConstraint, hence it should be unicode
11408
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
    92
    return text_type(build_index_name(table, attrs, 'unique_'))
11287
b537c07e3bdc pep8 schema2sql
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11005
diff changeset
    93
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    94
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
    95
def iter_unique_index_names(eschema):
11408
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
    96
    """Yield (attrs, index name) where attrs is a list of entity type's attribute names that should
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
    97
    be unique together, and index name the unique index name.
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
    98
    """
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
    99
    for attrs in eschema._unique_together or ():
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
   100
        yield attrs, unique_index_name(eschema, attrs)
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   101
11287
b537c07e3bdc pep8 schema2sql
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11005
diff changeset
   102
11789
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   103
def eschema_sql_def(dbhelper, eschema, skip_relations=(), prefix=''):
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   104
    """Return a list of (column names, sql type def) for the given entity schema.
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   105
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   106
    No constraint nor index are considered - this function is usually for massive import purpose.
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   107
    """
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   108
    attrs = [attrdef for attrdef in eschema.attribute_definitions()
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   109
             if not attrdef[0].type in skip_relations]
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   110
    attrs += [(rschema, None)
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   111
              for rschema in eschema.subject_relations()
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   112
              if not rschema.final and rschema.inlined]
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   113
    result = []
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   114
    for i in range(len(attrs)):
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   115
        rschema, attrschema = attrs[i]
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   116
        if attrschema is not None:
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   117
            # creating = False will avoid NOT NULL / REFERENCES constraints
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   118
            sqltype = aschema2sql(dbhelper, eschema, rschema, attrschema, creating=False)
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   119
        else:  # inline relation
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   120
            sqltype = 'integer'
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   121
        result.append(('%s%s' % (prefix, rschema.type), sqltype))
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   122
    return result
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   123
71df2811b422 [massive store] Store entities in temporary table as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11767
diff changeset
   124
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   125
def eschema2sql(dbhelper, eschema, skip_relations=(), prefix=''):
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   126
    """Yield SQL statements to initialize database from an entity schema."""
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   127
    table = prefix + eschema.type
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   128
    output = []
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   129
    w = output.append
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   130
    w('CREATE TABLE %s(' % (table))
11407
e6acdea9616c [schema2sql] inline eschema_attrs method
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11406
diff changeset
   131
    attrs = [attrdef for attrdef in eschema.attribute_definitions()
e6acdea9616c [schema2sql] inline eschema_attrs method
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11406
diff changeset
   132
             if not attrdef[0].type in skip_relations]
e6acdea9616c [schema2sql] inline eschema_attrs method
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11406
diff changeset
   133
    attrs += [(rschema, None)
e6acdea9616c [schema2sql] inline eschema_attrs method
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11406
diff changeset
   134
              for rschema in eschema.subject_relations()
e6acdea9616c [schema2sql] inline eschema_attrs method
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11406
diff changeset
   135
              if not rschema.final and rschema.inlined]
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   136
    for i in range(len(attrs)):
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   137
        rschema, attrschema = attrs[i]
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   138
        if attrschema is not None:
11409
218815f40576 [schema2sql] Drop unused indent argument on aschema2sql
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11408
diff changeset
   139
            sqltype = aschema2sql(dbhelper, eschema, rschema, attrschema)
11287
b537c07e3bdc pep8 schema2sql
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11005
diff changeset
   140
        else:  # inline relation
10201
989bbadbcd8d Add foreign key for inline relations
Julien Cristau <julien.cristau@logilab.fr>
parents: 10199
diff changeset
   141
            sqltype = 'integer REFERENCES entities (eid)'
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   142
        if i == len(attrs) - 1:
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   143
            w(' %s%s %s' % (prefix, rschema.type, sqltype))
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   144
        else:
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   145
            w(' %s%s %s,' % (prefix, rschema.type, sqltype))
10443
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   146
    for rschema, aschema in attrs:
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   147
        if aschema is None:  # inline relation
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   148
            continue
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   149
        rdef = rschema.rdef(eschema.type, aschema.type)
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   150
        for constraint in rdef.constraints:
11406
8ed625765a5c [schema2sql] Give a rdef to check_constraint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11362
diff changeset
   151
            cstrname, check = check_constraint(rdef, constraint, dbhelper, prefix=prefix)
10443
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   152
            if cstrname is not None:
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   153
                w(', CONSTRAINT %s CHECK(%s)' % (cstrname, check))
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   154
    w(')')
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   155
    yield '\n'.join(output)
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   156
    # create indexes
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   157
    for i in range(len(attrs)):
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   158
        rschema, attrschema = attrs[i]
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   159
        if attrschema is None or eschema.rdef(rschema).indexed:
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   160
            yield dbhelper.sql_create_index(table, prefix + rschema.type)
11359
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
   161
        if attrschema and any(isinstance(cstr, UniqueConstraint)
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
   162
                              for cstr in eschema.rdef(rschema).constraints):
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   163
            yield dbhelper.sql_create_index(table, prefix + rschema.type, unique=True)
11408
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
   164
    for attrs, index_name in iter_unique_index_names(eschema):
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
   165
        columns = ['%s%s' % (prefix, attr) for attr in attrs]
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
   166
        sqls = dbhelper.sqls_create_multicol_unique_index(table, columns, index_name)
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   167
        for sql in sqls:
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   168
            yield sql.rstrip(';')  # remove trailing ';' for consistency
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   169
11287
b537c07e3bdc pep8 schema2sql
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11005
diff changeset
   170
11410
5e449033adcc [schema2sql] Rename as_sql to constraint_value_as_sql
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11409
diff changeset
   171
def constraint_value_as_sql(value, dbhelper, prefix):
11408
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
   172
    """Return the SQL value from a Yams constraint's value, handling special cases where it's a
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
   173
    `Attribute`, `TODAY` or `NOW` instance instead of a literal value.
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
   174
    """
10444
fb7c1013189e [schema2sql] support NOW and TODAY in check constraints
Julien Cristau <julien.cristau@logilab.fr>
parents: 10443
diff changeset
   175
    if isinstance(value, Attribute):
fb7c1013189e [schema2sql] support NOW and TODAY in check constraints
Julien Cristau <julien.cristau@logilab.fr>
parents: 10443
diff changeset
   176
        return prefix + value.attr
fb7c1013189e [schema2sql] support NOW and TODAY in check constraints
Julien Cristau <julien.cristau@logilab.fr>
parents: 10443
diff changeset
   177
    elif isinstance(value, TODAY):
fb7c1013189e [schema2sql] support NOW and TODAY in check constraints
Julien Cristau <julien.cristau@logilab.fr>
parents: 10443
diff changeset
   178
        return dbhelper.sql_current_date()
fb7c1013189e [schema2sql] support NOW and TODAY in check constraints
Julien Cristau <julien.cristau@logilab.fr>
parents: 10443
diff changeset
   179
    elif isinstance(value, NOW):
fb7c1013189e [schema2sql] support NOW and TODAY in check constraints
Julien Cristau <julien.cristau@logilab.fr>
parents: 10443
diff changeset
   180
        return dbhelper.sql_current_timestamp()
fb7c1013189e [schema2sql] support NOW and TODAY in check constraints
Julien Cristau <julien.cristau@logilab.fr>
parents: 10443
diff changeset
   181
    else:
fb7c1013189e [schema2sql] support NOW and TODAY in check constraints
Julien Cristau <julien.cristau@logilab.fr>
parents: 10443
diff changeset
   182
        # XXX more quoting for literals?
fb7c1013189e [schema2sql] support NOW and TODAY in check constraints
Julien Cristau <julien.cristau@logilab.fr>
parents: 10443
diff changeset
   183
        return value
fb7c1013189e [schema2sql] support NOW and TODAY in check constraints
Julien Cristau <julien.cristau@logilab.fr>
parents: 10443
diff changeset
   184
11287
b537c07e3bdc pep8 schema2sql
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11005
diff changeset
   185
11406
8ed625765a5c [schema2sql] Give a rdef to check_constraint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11362
diff changeset
   186
def check_constraint(rdef, constraint, dbhelper, prefix=''):
8ed625765a5c [schema2sql] Give a rdef to check_constraint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11362
diff changeset
   187
    """Return (constraint name, constraint SQL definition) for the given relation definition's
8ed625765a5c [schema2sql] Give a rdef to check_constraint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11362
diff changeset
   188
    constraint. Maybe (None, None) if the constraint is not handled in the backend.
8ed625765a5c [schema2sql] Give a rdef to check_constraint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11362
diff changeset
   189
    """
8ed625765a5c [schema2sql] Give a rdef to check_constraint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11362
diff changeset
   190
    attr = rdef.rtype.type
11416
9c2fbb872e91 [schema] Add a method on yams constraints to compute its unique name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11413
diff changeset
   191
    cstrname = constraint.name_for(rdef)
10443
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   192
    if constraint.type() == 'BoundaryConstraint':
11410
5e449033adcc [schema2sql] Rename as_sql to constraint_value_as_sql
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11409
diff changeset
   193
        value = constraint_value_as_sql(constraint.boundary, dbhelper, prefix)
10443
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   194
        return cstrname, '%s%s %s %s' % (prefix, attr, constraint.operator, value)
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   195
    elif constraint.type() == 'IntervalBoundConstraint':
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   196
        condition = []
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   197
        if constraint.minvalue is not None:
11410
5e449033adcc [schema2sql] Rename as_sql to constraint_value_as_sql
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11409
diff changeset
   198
            value = constraint_value_as_sql(constraint.minvalue, dbhelper, prefix)
10443
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   199
            condition.append('%s%s >= %s' % (prefix, attr, value))
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   200
        if constraint.maxvalue is not None:
11410
5e449033adcc [schema2sql] Rename as_sql to constraint_value_as_sql
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11409
diff changeset
   201
            value = constraint_value_as_sql(constraint.maxvalue, dbhelper, prefix)
10443
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   202
            condition.append('%s%s <= %s' % (prefix, attr, value))
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   203
        return cstrname, ' AND '.join(condition)
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   204
    elif constraint.type() == 'StaticVocabularyConstraint':
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   205
        sample = next(iter(constraint.vocabulary()))
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   206
        if not isinstance(sample, string_types):
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   207
            values = ', '.join(str(word) for word in constraint.vocabulary())
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   208
        else:
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   209
            # XXX better quoting?
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   210
            values = ', '.join("'%s'" % word.replace("'", "''") for word in constraint.vocabulary())
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   211
        return cstrname, '%s%s IN (%s)' % (prefix, attr, values)
2d3834df64ab [schema2sql] insert some constraints into the backend
Julien Cristau <julien.cristau@logilab.fr>
parents: 10204
diff changeset
   212
    return None, None
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   213
11287
b537c07e3bdc pep8 schema2sql
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11005
diff changeset
   214
11409
218815f40576 [schema2sql] Drop unused indent argument on aschema2sql
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11408
diff changeset
   215
def aschema2sql(dbhelper, eschema, rschema, aschema, creating=True):
11408
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
   216
    """Return string containing a SQL table's column definition from attribute schema."""
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   217
    attr = rschema.type
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   218
    rdef = rschema.rdef(eschema.type, aschema.type)
11359
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
   219
    sqltype = type_from_rdef(dbhelper, rdef)
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   220
    if SET_DEFAULT:
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   221
        default = eschema.default(attr)
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   222
        if default is not None:
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   223
            if aschema.type == 'Boolean':
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   224
                sqltype += ' DEFAULT %s' % dbhelper.boolean_value(default)
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   225
            elif aschema.type == 'String':
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   226
                sqltype += ' DEFAULT %r' % str(default)
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   227
            elif aschema.type in ('Int', 'BigInt', 'Float'):
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   228
                sqltype += ' DEFAULT %s' % default
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   229
            # XXX ignore default for other type
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   230
            # this is expected for NOW / TODAY
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   231
    if creating:
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   232
        if rdef.uid:
10204
f8ccae1e271d Add reference from etype table's eid column to the entities table
Julien Cristau <julien.cristau@logilab.fr>
parents: 10202
diff changeset
   233
            sqltype += ' PRIMARY KEY REFERENCES entities (eid)'
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   234
        elif rdef.cardinality[0] == '1':
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   235
            # don't set NOT NULL if backend isn't able to change it later
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   236
            if dbhelper.alter_column_support:
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   237
                sqltype += ' NOT NULL'
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   238
    # else we're getting sql type to alter a column, we don't want key / indexes
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   239
    # / null modifiers
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   240
    return sqltype
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   241
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   242
11359
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
   243
def type_from_rdef(dbhelper, rdef):
11408
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
   244
    """Return a string containing SQL type name for the given relation definition."""
11005
f8417bd135ed [server, hooks] allow callable in dbh.TYPE_MAPPING
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11003
diff changeset
   245
    constraints = list(rdef.constraints)
11359
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
   246
    sqltype = None
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
   247
    if rdef.object.type == 'String':
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
   248
        for constraint in constraints:
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
   249
            if isinstance(constraint, SizeConstraint) and constraint.max is not None:
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
   250
                size_constrained_string = dbhelper.TYPE_MAPPING.get(
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
   251
                    'SizeConstrainedString', 'varchar(%s)')
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
   252
                sqltype = size_constrained_string % constraint.max
2da2dd60331c [sql gen] Explicitly name unique index
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11291
diff changeset
   253
                break
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   254
    if sqltype is None:
11005
f8417bd135ed [server, hooks] allow callable in dbh.TYPE_MAPPING
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11003
diff changeset
   255
        sqltype = sql_type(dbhelper, rdef)
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   256
    return sqltype
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   257
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   258
11005
f8417bd135ed [server, hooks] allow callable in dbh.TYPE_MAPPING
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11003
diff changeset
   259
def sql_type(dbhelper, rdef):
11408
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
   260
    """Return a string containing SQL type to use to store values of the given relation definition.
5be298ed4caa [schema2sql] More docstrings and minor API improvements
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11407
diff changeset
   261
    """
11005
f8417bd135ed [server, hooks] allow callable in dbh.TYPE_MAPPING
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11003
diff changeset
   262
    sqltype = dbhelper.TYPE_MAPPING[rdef.object]
f8417bd135ed [server, hooks] allow callable in dbh.TYPE_MAPPING
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11003
diff changeset
   263
    if callable(sqltype):
f8417bd135ed [server, hooks] allow callable in dbh.TYPE_MAPPING
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11003
diff changeset
   264
        sqltype = sqltype(rdef)
f8417bd135ed [server, hooks] allow callable in dbh.TYPE_MAPPING
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11003
diff changeset
   265
    return sqltype
f8417bd135ed [server, hooks] allow callable in dbh.TYPE_MAPPING
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11003
diff changeset
   266
f8417bd135ed [server, hooks] allow callable in dbh.TYPE_MAPPING
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11003
diff changeset
   267
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   268
_SQL_SCHEMA = """
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   269
CREATE TABLE %(table)s (
10202
aaabcb64f77f Use foreign keys for relations tables
Julien Cristau <julien.cristau@logilab.fr>
parents: 10201
diff changeset
   270
  eid_from INTEGER NOT NULL REFERENCES entities (eid),
aaabcb64f77f Use foreign keys for relations tables
Julien Cristau <julien.cristau@logilab.fr>
parents: 10201
diff changeset
   271
  eid_to INTEGER NOT NULL REFERENCES entities (eid),
11360
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
   272
  CONSTRAINT %(pkey_idx)s PRIMARY KEY(eid_from, eid_to)
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   273
);
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   274
11360
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
   275
CREATE INDEX %(from_idx)s ON %(table)s(eid_from);
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   276
CREATE INDEX %(to_idx)s ON %(table)s(eid_to)"""
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   277
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   278
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   279
def rschema2sql(rschema):
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   280
    """Yield SQL statements to create database table and indexes for a Yams relation schema."""
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   281
    assert not rschema.rule
11360
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
   282
    table = '%s_relation' % rschema.type
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   283
    sqls = _SQL_SCHEMA % {'table': table,
11360
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
   284
                          'pkey_idx': build_index_name(table, ['eid_from', 'eid_to'], 'key_'),
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
   285
                          'from_idx': build_index_name(table, ['eid_from'], 'idx_'),
49aca289134f [sql gen] Control size of index name using an md5 hash
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11359
diff changeset
   286
                          'to_idx': build_index_name(table, ['eid_to'], 'idx_')}
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   287
    for sql in sqls.split(';'):
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   288
        yield sql.strip()
10481
6ac4b1726e9f [schema2sql] properly consider skip_relations parameters.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10444
diff changeset
   289
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   290
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   291
def grant_schema(schema, user, set_owner=True, skip_entities=(), prefix=''):
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   292
    """Yield SQL statements to give all access (and ownership if `set_owner` is True) on the
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   293
    database tables for the given Yams schema to `user`.
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   294
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   295
    `prefix` may be a string that will be prepended to all table / column names (usually, 'cw_').
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   296
    """
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   297
    for etype in sorted(schema.entities()):
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   298
        eschema = schema.eschema(etype)
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   299
        if eschema.final or etype in skip_entities:
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   300
            continue
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   301
        for sql in grant_eschema(eschema, user, set_owner, prefix=prefix):
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   302
            yield sql
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   303
    for rtype in sorted(schema.relations()):
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   304
        rschema = schema.rschema(rtype)
10481
6ac4b1726e9f [schema2sql] properly consider skip_relations parameters.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10444
diff changeset
   305
        if rschema_has_table(rschema, skip_relations=()):  # XXX skip_relations should be specified
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   306
            for sql in grant_rschema(rschema, user, set_owner):
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   307
                yield sql
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   308
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   309
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   310
def grant_eschema(eschema, user, set_owner=True, prefix=''):
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   311
    """Yield SQL statements to give all access (and ownership if `set_owner` is True) on the
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   312
    database tables for the given Yams entity schema to `user`.
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   313
    """
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   314
    etype = eschema.type
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   315
    if set_owner:
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   316
        yield 'ALTER TABLE %s%s OWNER TO %s' % (prefix, etype, user)
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   317
    yield 'GRANT ALL ON %s%s TO %s' % (prefix, etype, user)
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   318
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   319
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   320
def grant_rschema(rschema, user, set_owner=True):
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   321
    """Yield SQL statements to give all access (and ownership if `set_owner` is True) on the
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   322
    database tables for the given Yams relation schema to `user`.
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   323
    """
10199
218c28bff695 Steal schema2sql module from yams
Julien Cristau <julien.cristau@logilab.fr>
parents:
diff changeset
   324
    if set_owner:
11413
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   325
        yield 'ALTER TABLE %s_relation OWNER TO %s' % (rschema.type, user)
c172fa18565e [schema2sql] Avoid "parsing" SQL statements for database initialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11412
diff changeset
   326
    yield 'GRANT ALL ON %s_relation TO %s' % (rschema.type, user)