server/sources/native.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Mon, 29 Mar 2010 13:34:24 +0200
branchstable
changeset 5061 cdab5220eac0
parent 5059 1d5c81588144
child 5067 adc2122eed03
permissions -rw-r--r--
[cleanup]
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     1
"""Adapters for native cubicweb sources.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     2
1952
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
     3
Notes:
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
     4
* extid (aka external id, the primary key of an entity in the external source
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
     5
  from which it comes from) are stored in a varchar column encoded as a base64
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
     6
  string. This is because it should actually be Bytes but we want an index on
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
     7
  it for fast querying.
2056
57d287a2132a reminder
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
     8
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     9
:organization: Logilab
4212
ab6573088b4a update copyright: welcome 2010
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4198
diff changeset
    10
:copyright: 2001-2010 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    11
:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr
1977
606923dff11b big bunch of copyright / docstring update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1954
diff changeset
    12
:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    13
"""
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
    14
from __future__ import with_statement
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
    15
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    16
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    17
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
    18
from pickle import loads, dumps
4900
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4893
diff changeset
    19
from threading import Lock
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 973
diff changeset
    20
from datetime import datetime
1952
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
    21
from base64 import b64decode, b64encode
4964
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
    22
from contextlib import contextmanager
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
    24
from logilab.common.compat import any
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
from logilab.common.cache import Cache
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
    26
from logilab.common.decorators import cached, clear_cache
3835
a191b3b9e455 more sensible default values to c-c "create" inputs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    27
from logilab.common.configuration import Method
a191b3b9e455 more sensible default values to c-c "create" inputs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    28
from logilab.common.shellutils import getlogin
4848
41f84eea63c9 rename logilab.db into logilab.database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4845
diff changeset
    29
from logilab.database import get_db_helper
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    30
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
    31
from cubicweb import UnknownEid, AuthenticationError, Binary, server, neg_role
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
    32
from cubicweb import transaction as tx
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
    33
from cubicweb.schema import VIRTUAL_RTYPES
3835
a191b3b9e455 more sensible default values to c-c "create" inputs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    34
from cubicweb.cwconfig import CubicWebNoAppConfiguration
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
    35
from cubicweb.server import hook
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    36
from cubicweb.server.utils import crypt_password
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2638
diff changeset
    37
from cubicweb.server.sqlutils import SQL_PREFIX, SQLAdapterMixIn
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    38
from cubicweb.server.rqlannotation import set_qdata
4943
7f5b83578fec disable security when undoing
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4913
diff changeset
    39
from cubicweb.server.session import hooks_control, security_enabled
2625
d6012db7b93e R [server debug] more server side debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2620
diff changeset
    40
from cubicweb.server.sources import AbstractSource, dbg_st_search, dbg_results
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    41
from cubicweb.server.sources.rql2sql import SQLGenerator
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    42
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    43
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2310
diff changeset
    44
ATTR_MAP = {}
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    45
NONSYSTEM_ETYPES = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    46
NONSYSTEM_RELATIONS = set()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    47
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    48
class LogCursor(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    49
    def __init__(self, cursor):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
        self.cu = cursor
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
    51
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
    def execute(self, query, args=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    53
        """Execute a query.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    54
        it's a function just so that it shows up in profiling
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    55
        """
2593
16d9419a4a79 F: start to handle binary debug log level on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2589
diff changeset
    56
        if server.DEBUG & server.DBG_SQL:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    57
            print 'exec', query, args
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    58
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    59
            self.cu.execute(str(query), args)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    60
        except Exception, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    61
            print "sql: %r\n args: %s\ndbms message: %r" % (
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    62
                query, args, ex.args[0])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
            raise
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
    64
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    65
    def fetchall(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    66
        return self.cu.fetchall()
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
    67
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    68
    def fetchone(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    69
        return self.cu.fetchone()
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
    70
2625
d6012db7b93e R [server debug] more server side debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2620
diff changeset
    71
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    72
def make_schema(selected, solution, table, typemap):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    73
    """return a sql schema to store RQL query result"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    74
    sql = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    75
    varmap = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    76
    for i, term in enumerate(selected):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    77
        name = 'C%s' % i
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    78
        key = term.as_string()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    79
        varmap[key] = '%s.%s' % (table, name)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    80
        ttype = term.get_type(solution)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    81
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    82
            sql.append('%s %s' % (name, typemap[ttype]))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    83
        except KeyError:
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3497
diff changeset
    84
            # assert not schema(ttype).final
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    85
            sql.append('%s %s' % (name, typemap['Int']))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
    return ','.join(sql), varmap
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
2625
d6012db7b93e R [server debug] more server side debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2620
diff changeset
    88
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    89
def _modified_sql(table, etypes):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
    # XXX protect against sql injection
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
    if len(etypes) > 1:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
        restr = 'type IN (%s)' % ','.join("'%s'" % etype for etype in etypes)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    94
        restr = "type='%s'" % etypes[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
    if table == 'entities':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
        attr = 'mtime'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    97
    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    98
        attr = 'dtime'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    99
    return 'SELECT type, eid FROM %s WHERE %s AND %s > %%(time)s' % (
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   100
        table, restr, attr)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   101
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   102
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   103
def sql_or_clauses(sql, clauses):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   104
    select, restr = sql.split(' WHERE ', 1)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   105
    restrclauses = restr.split(' AND ')
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   106
    for clause in clauses:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   107
        restrclauses.remove(clause)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   108
    if restrclauses:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   109
        restr = '%s AND (%s)' % (' AND '.join(restrclauses),
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   110
                                 ' OR '.join(clauses))
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   111
    else:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   112
        restr = '(%s)' % ' OR '.join(clauses)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   113
    return '%s WHERE %s' % (select, restr)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   114
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   115
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   116
class UndoException(Exception):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   117
    """something went wrong during undoing"""
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   118
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   119
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   120
def _undo_check_relation_target(tentity, rdef, role):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   121
    """check linked entity has not been redirected for this relation"""
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   122
    card = rdef.role_cardinality(role)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   123
    if card in '?1' and tentity.related(rdef.rtype, role):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   124
        raise UndoException(tentity._cw._(
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   125
            "Can't restore %(role)s relation %(rtype)s to entity %(eid)s which "
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   126
            "is already linked using this relation.")
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   127
                            % {'role': neg_role(role),
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   128
                               'rtype': rdef.rtype,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   129
                               'eid': tentity.eid})
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   130
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   131
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   132
class NativeSQLSource(SQLAdapterMixIn, AbstractSource):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   133
    """adapter for source using the native cubicweb schema (see below)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   134
    """
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2310
diff changeset
   135
    sqlgen_class = SQLGenerator
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
    options = (
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
        ('db-driver',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   138
         {'type' : 'string',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   139
          'default': 'postgres',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   140
          'help': 'database driver (postgres or sqlite)',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
          'group': 'native-source', 'inputlevel': 1,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
          }),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   143
        ('db-host',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   144
         {'type' : 'string',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   145
          'default': '',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   146
          'help': 'database host',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   147
          'group': 'native-source', 'inputlevel': 1,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   148
          }),
2566
714a8743d423 missing db-port option to source's option definitions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2354
diff changeset
   149
        ('db-port',
714a8743d423 missing db-port option to source's option definitions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2354
diff changeset
   150
         {'type' : 'string',
714a8743d423 missing db-port option to source's option definitions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2354
diff changeset
   151
          'default': '',
714a8743d423 missing db-port option to source's option definitions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2354
diff changeset
   152
          'help': 'database port',
714a8743d423 missing db-port option to source's option definitions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2354
diff changeset
   153
          'group': 'native-source', 'inputlevel': 1,
714a8743d423 missing db-port option to source's option definitions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2354
diff changeset
   154
          }),
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   155
        ('db-name',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   156
         {'type' : 'string',
3835
a191b3b9e455 more sensible default values to c-c "create" inputs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   157
          'default': Method('default_instance_id'),
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   158
          'help': 'database name',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   159
          'group': 'native-source', 'inputlevel': 0,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
          }),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   161
        ('db-user',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   162
         {'type' : 'string',
3835
a191b3b9e455 more sensible default values to c-c "create" inputs
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   163
          'default': CubicWebNoAppConfiguration.mode == 'user' and getlogin() or 'cubicweb',
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   164
          'help': 'database user',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   165
          'group': 'native-source', 'inputlevel': 0,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   166
          }),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   167
        ('db-password',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   168
         {'type' : 'password',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   169
          'default': '',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   170
          'help': 'database password',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   171
          'group': 'native-source', 'inputlevel': 0,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   172
          }),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   173
        ('db-encoding',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   174
         {'type' : 'string',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   175
          'default': 'utf8',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   176
          'help': 'database encoding',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   177
          'group': 'native-source', 'inputlevel': 1,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   178
          }),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   179
    )
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   180
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   181
    def __init__(self, repo, appschema, source_config, *args, **kwargs):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   182
        SQLAdapterMixIn.__init__(self, source_config)
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   183
        self.authentifiers = [LoginPasswordAuthentifier(self)]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   184
        AbstractSource.__init__(self, repo, appschema, source_config,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   185
                                *args, **kwargs)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   186
        # sql generator
2354
9b4bac626977 ability to map attributes to something else than usual cw mapping on sql generation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2310
diff changeset
   187
        self._rql_sqlgen = self.sqlgen_class(appschema, self.dbhelper,
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4829
diff changeset
   188
                                             ATTR_MAP.copy())
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   189
        # full text index helper
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   190
        self.do_fti = not repo.config['delay-full-text-indexation']
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   191
        # sql queries cache
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   192
        self._cache = Cache(repo.config['rql-cache-size'])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   193
        self._temp_table_data = {}
5061
cdab5220eac0 [cleanup]
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5059
diff changeset
   194
        # we need a lock to protect eid attribution function (XXX, really?
cdab5220eac0 [cleanup]
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5059
diff changeset
   195
        # explain)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   196
        self._eid_creation_lock = Lock()
4964
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   197
        # (etype, attr) / storage mapping
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   198
        self._storages = {}
5059
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   199
        # entity types that may be used by other multi-sources instances
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   200
        self.multisources_etypes = set(repo.config['multi-sources-etypes'])
2072
8008e8812d76 deactivate sqlite connection wrapping for unittest_multisources for now
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2066
diff changeset
   201
        # XXX no_sqlite_wrap trick since we've a sqlite locking pb when
8008e8812d76 deactivate sqlite connection wrapping for unittest_multisources for now
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2066
diff changeset
   202
        # running unittest_multisources with the wrapping below
8008e8812d76 deactivate sqlite connection wrapping for unittest_multisources for now
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2066
diff changeset
   203
        if self.dbdriver == 'sqlite' and \
8008e8812d76 deactivate sqlite connection wrapping for unittest_multisources for now
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2066
diff changeset
   204
               not getattr(repo.config, 'no_sqlite_wrap', False):
2064
a5cd3a92314a properly call [re]set_pool, fix connection handling so we have a change to get cw running on top of a sqlite dabase with threads activated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2056
diff changeset
   205
            from cubicweb.server.sources.extlite import ConnectionWrapper
a5cd3a92314a properly call [re]set_pool, fix connection handling so we have a change to get cw running on top of a sqlite dabase with threads activated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2056
diff changeset
   206
            self.get_connection = lambda: ConnectionWrapper(self)
a5cd3a92314a properly call [re]set_pool, fix connection handling so we have a change to get cw running on top of a sqlite dabase with threads activated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2056
diff changeset
   207
            self.check_connection = lambda cnx: cnx
a5cd3a92314a properly call [re]set_pool, fix connection handling so we have a change to get cw running on top of a sqlite dabase with threads activated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2056
diff changeset
   208
            def pool_reset(cnx):
2620
de68f84b8f54 R [sql source] cnx._cnx checked in cnx.close, don't do it here
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2619
diff changeset
   209
                cnx.close()
2064
a5cd3a92314a properly call [re]set_pool, fix connection handling so we have a change to get cw running on top of a sqlite dabase with threads activated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2056
diff changeset
   210
            self.pool_reset = pool_reset
a5cd3a92314a properly call [re]set_pool, fix connection handling so we have a change to get cw running on top of a sqlite dabase with threads activated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2056
diff changeset
   211
a5cd3a92314a properly call [re]set_pool, fix connection handling so we have a change to get cw running on top of a sqlite dabase with threads activated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2056
diff changeset
   212
    @property
a5cd3a92314a properly call [re]set_pool, fix connection handling so we have a change to get cw running on top of a sqlite dabase with threads activated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2056
diff changeset
   213
    def _sqlcnx(self):
a5cd3a92314a properly call [re]set_pool, fix connection handling so we have a change to get cw running on top of a sqlite dabase with threads activated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2056
diff changeset
   214
        # XXX: sqlite connections can only be used in the same thread, so
a5cd3a92314a properly call [re]set_pool, fix connection handling so we have a change to get cw running on top of a sqlite dabase with threads activated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2056
diff changeset
   215
        #      create a new one each time necessary. If it appears to be time
a5cd3a92314a properly call [re]set_pool, fix connection handling so we have a change to get cw running on top of a sqlite dabase with threads activated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2056
diff changeset
   216
        #      consuming, find another way
a5cd3a92314a properly call [re]set_pool, fix connection handling so we have a change to get cw running on top of a sqlite dabase with threads activated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2056
diff changeset
   217
        return SQLAdapterMixIn.get_connection(self)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   218
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   219
    def add_authentifier(self, authentifier):
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   220
        self.authentifiers.append(authentifier)
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   221
        authentifier.source = self
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   222
        authentifier.set_schema(self.schema)
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   223
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   224
    def reset_caches(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   225
        """method called during test to reset potential source caches"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   226
        self._cache = Cache(self.repo.config['rql-cache-size'])
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   227
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   228
    def clear_eid_cache(self, eid, etype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   229
        """clear potential caches for the given eid"""
2610
2933cc6bf9ad [F native source] fix attempts to clear cache key
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2593
diff changeset
   230
        self._cache.pop('Any X WHERE X eid %s, X is %s' % (eid, etype), None)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   231
        self._cache.pop('Any X WHERE X eid %s' % eid, None)
2610
2933cc6bf9ad [F native source] fix attempts to clear cache key
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2593
diff changeset
   232
        self._cache.pop('Any %s' % eid, None)
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   233
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   234
    def sqlexec(self, session, sql, args=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   235
        """execute the query and return its result"""
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   236
        return self.process_result(self.doexec(session, sql, args))
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   237
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   238
    def init_creating(self):
2064
a5cd3a92314a properly call [re]set_pool, fix connection handling so we have a change to get cw running on top of a sqlite dabase with threads activated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2056
diff changeset
   239
        pool = self.repo._get_pool()
a5cd3a92314a properly call [re]set_pool, fix connection handling so we have a change to get cw running on top of a sqlite dabase with threads activated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2056
diff changeset
   240
        pool.pool_set()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   241
        # check full text index availibility
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   242
        if self.do_fti:
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4829
diff changeset
   243
            if not self.dbhelper.has_fti_table(pool['system']):
4825
cdd979ae1b57 fix name error fixed in wrong branch
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4818
diff changeset
   244
                if not self.repo.config.creating:
4810
3055dc7a7c88 don't issue critical warning about missing text index when we're creating the instance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4807
diff changeset
   245
                    self.critical('no text index table')
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   246
                self.do_fti = False
2064
a5cd3a92314a properly call [re]set_pool, fix connection handling so we have a change to get cw running on top of a sqlite dabase with threads activated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2056
diff changeset
   247
        pool.pool_reset()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   248
        self.repo._free_pool(pool)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   249
4893
15ae9a33a7f2 [db backup] fix name error in backup_to_file: we've to pass .confirm all along the chain as for restore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4836
diff changeset
   250
    def backup(self, backupfile, confirm):
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2638
diff changeset
   251
        """method called to create a backup of the source's data"""
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2638
diff changeset
   252
        self.close_pool_connections()
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2638
diff changeset
   253
        try:
4893
15ae9a33a7f2 [db backup] fix name error in backup_to_file: we've to pass .confirm all along the chain as for restore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4836
diff changeset
   254
            self.backup_to_file(backupfile, confirm)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2638
diff changeset
   255
        finally:
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2638
diff changeset
   256
            self.open_pool_connections()
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   257
4195
86dcaf6bb92f closes #601987
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4185
diff changeset
   258
    def restore(self, backupfile, confirm, drop):
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   259
        """method called to restore a backup of source's data"""
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2759
diff changeset
   260
        if self.repo.config.open_connections_pools:
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2759
diff changeset
   261
            self.close_pool_connections()
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2638
diff changeset
   262
        try:
4195
86dcaf6bb92f closes #601987
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4185
diff changeset
   263
            self.restore_from_file(backupfile, confirm, drop=drop)
2759
23d7a75693f8 R refactor backup and use tar.gz to store all sources
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 2638
diff changeset
   264
        finally:
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2759
diff changeset
   265
            if self.repo.config.open_connections_pools:
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2759
diff changeset
   266
                self.open_pool_connections()
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   267
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   268
    def init(self):
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   269
        self.init_creating()
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   270
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5011
diff changeset
   271
    # XXX deprecates [un]map_attribute ?
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5011
diff changeset
   272
    def map_attribute(self, etype, attr, cb, sourcedb=True):
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5011
diff changeset
   273
        self._rql_sqlgen.attr_map['%s.%s' % (etype, attr)] = (cb, sourcedb)
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   274
4512
e7ac20bf3629 unset_attribute_storage, for testing purpose at least
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   275
    def unmap_attribute(self, etype, attr):
e7ac20bf3629 unset_attribute_storage, for testing purpose at least
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   276
        self._rql_sqlgen.attr_map.pop('%s.%s' % (etype, attr), None)
e7ac20bf3629 unset_attribute_storage, for testing purpose at least
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
   277
4964
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   278
    def set_storage(self, etype, attr, storage):
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   279
        storage_dict = self._storages.setdefault(etype, {})
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   280
        storage_dict[attr] = storage
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5011
diff changeset
   281
        self.map_attribute(etype, attr,
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5011
diff changeset
   282
                           storage.callback, storage.is_source_callback)
4964
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   283
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   284
    def unset_storage(self, etype, attr):
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   285
        self._storages[etype].pop(attr)
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   286
        # if etype has no storage left, remove the entry
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   287
        if not self._storages[etype]:
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   288
            del self._storages[etype]
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   289
        self.unmap_attribute(etype, attr)
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   290
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   291
    # ISource interface #######################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   292
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   293
    def compile_rql(self, rql, sols):
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3041
diff changeset
   294
        rqlst = self.repo.vreg.rqlhelper.parse(rql)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   295
        rqlst.restricted_vars = ()
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   296
        rqlst.children[0].solutions = sols
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   297
        self.repo.querier.sqlgen_annotate(rqlst)
438
69b79faefa94 need_intersect test and fixes
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   298
        set_qdata(self.schema.rschema, rqlst, ())
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   299
        return rqlst
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   300
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   301
    def set_schema(self, schema):
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2354
diff changeset
   302
        """set the instance'schema"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   303
        self._cache = Cache(self.repo.config['rql-cache-size'])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   304
        self.cache_hit, self.cache_miss, self.no_cache = 0, 0, 0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   305
        self.schema = schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   306
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   307
            self._rql_sqlgen.schema = schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   308
        except AttributeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   309
            pass # __init__
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   310
        for authentifier in self.authentifiers:
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   311
            authentifier.set_schema(self.schema)
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   312
        clear_cache(self, 'need_fti_indexation')
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   313
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
    def support_entity(self, etype, write=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   315
        """return true if the given entity's type is handled by this adapter
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   316
        if write is true, return true only if it's a RW support
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   317
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   318
        return not etype in NONSYSTEM_ETYPES
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   319
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   320
    def support_relation(self, rtype, write=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
        """return true if the given relation's type is handled by this adapter
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   322
        if write is true, return true only if it's a RW support
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
        if write:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
            return not rtype in NONSYSTEM_RELATIONS
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
        # due to current multi-sources implementation, the system source
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   327
        # can't claim not supporting a relation
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
        return True #not rtype == 'content_for'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
3041
782fa7566a22 [multi-sources] new may_cross_relation method on sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
   330
    def may_cross_relation(self, rtype):
782fa7566a22 [multi-sources] new may_cross_relation method on sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
   331
        return True
782fa7566a22 [multi-sources] new may_cross_relation method on sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
   332
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   333
    def authenticate(self, session, login, **kwargs):
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   334
        """return CWUser eid for the given login and other authentication
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   335
        information found in kwargs, else raise `AuthenticationError`
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   336
        """
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   337
        for authentifier in self.authentifiers:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   338
            try:
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   339
                return authentifier.authenticate(session, login, **kwargs)
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   340
            except AuthenticationError:
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   341
                continue
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
   342
        raise AuthenticationError()
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   343
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   344
    def syntax_tree_search(self, session, union, args=None, cachekey=None,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
                           varmap=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   346
        """return result from this source for a rql query (actually from
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   347
        a rql syntax tree and a solution dictionary mapping each used
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   348
        variable to a possible type). If cachekey is given, the query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
        necessary to fetch the results (but not the results themselves)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   350
        may be cached using this key.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   351
        """
2625
d6012db7b93e R [server debug] more server side debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2620
diff changeset
   352
        assert dbg_st_search(self.uri, union, varmap, args, cachekey)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   353
        # remember number of actually selected term (sql generation may append some)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   354
        if cachekey is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   355
            self.no_cache += 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   356
            # generate sql query if we are able to do so (not supported types...)
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5011
diff changeset
   357
            sql, qargs, cbs = self._rql_sqlgen.generate(union, args, varmap)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   358
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   359
            # sql may be cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   360
            try:
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5011
diff changeset
   361
                sql, qargs, cbs = self._cache[cachekey]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   362
                self.cache_hit += 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   363
            except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   364
                self.cache_miss += 1
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5011
diff changeset
   365
                sql, qargs, cbs = self._rql_sqlgen.generate(union, args, varmap)
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5011
diff changeset
   366
                self._cache[cachekey] = sql, qargs, cbs
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5011
diff changeset
   367
        args = self.merge_args(args, qargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   368
        assert isinstance(sql, basestring), repr(sql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   369
        try:
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   370
            cursor = self.doexec(session, sql, args)
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4829
diff changeset
   371
        except (self.OperationalError, self.InterfaceError):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   372
            # FIXME: better detection of deconnection pb
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   373
            self.info("request failed '%s' ... retry with a new cursor", sql)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   374
            session.pool.reconnect(self)
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   375
            cursor = self.doexec(session, sql, args)
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5011
diff changeset
   376
        results = self.process_result(cursor, cbs)
2625
d6012db7b93e R [server debug] more server side debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2620
diff changeset
   377
        assert dbg_results(results)
d6012db7b93e R [server debug] more server side debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2620
diff changeset
   378
        return results
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   379
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   380
    def flying_insert(self, table, session, union, args=None, varmap=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   381
        """similar as .syntax_tree_search, but inserts data in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   382
        temporary table (on-the-fly if possible, eg for the system
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   383
        source whose the given cursor come from). If not possible,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   384
        inserts all data by calling .executemany().
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
        """
2625
d6012db7b93e R [server debug] more server side debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2620
diff changeset
   386
        assert dbg_st_search(
2638
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2627
diff changeset
   387
            self.uri, union, varmap, args,
2625
d6012db7b93e R [server debug] more server side debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2620
diff changeset
   388
            prefix='ON THE FLY temp data insertion into %s from' % table)
d6012db7b93e R [server debug] more server side debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2620
diff changeset
   389
        # generate sql queries if we are able to do so
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5011
diff changeset
   390
        sql, qargs, cbs = self._rql_sqlgen.generate(union, args, varmap)
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4829
diff changeset
   391
        query = 'INSERT INTO %s %s' % (table, sql.encode(self._dbencoding))
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5011
diff changeset
   392
        self.doexec(session, query, self.merge_args(args, qargs))
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   393
2627
d710278e0c1c manual_insert is a public method
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2625
diff changeset
   394
    def manual_insert(self, results, table, session):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   395
        """insert given result into a temporary table on the system source"""
2625
d6012db7b93e R [server debug] more server side debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2620
diff changeset
   396
        if server.DEBUG & server.DBG_RQL:
3958
505025eb0d37 fix name error while debugging
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3835
diff changeset
   397
            print '  manual insertion of', results, 'into', table
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   398
        if not results:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   399
            return
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   400
        query_args = ['%%(%s)s' % i for i in xrange(len(results[0]))]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
        query = 'INSERT INTO %s VALUES(%s)' % (table, ','.join(query_args))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   402
        kwargs_list = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   403
        for row in results:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   404
            kwargs = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   405
            row = tuple(row)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   406
            for index, cell in enumerate(row):
2066
2c4bf4ee88a2 cleanup, stop encoding unicode string in manual_insert, no more necessary and make crash recent sqlite w/ 8bit string
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2064
diff changeset
   407
                if isinstance(cell, Binary):
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4829
diff changeset
   408
                    cell = self._binary(cell.getvalue())
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   409
                kwargs[str(index)] = cell
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   410
            kwargs_list.append(kwargs)
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   411
        self.doexecmany(session, query, kwargs_list)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   412
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   413
    def clean_temp_data(self, session, temptables):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   414
        """remove temporary data, usually associated to temporary tables"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   415
        if temptables:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   416
            for table in temptables:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
                try:
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   418
                    self.doexec(session,'DROP TABLE %s' % table)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   419
                except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   420
                    pass
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
                try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   422
                    del self._temp_table_data[table]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
                except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   424
                    continue
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   425
4964
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   426
    @contextmanager
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   427
    def _storage_handler(self, entity, event):
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   428
        # 1/ memorize values as they are before the storage is called.
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   429
        #    For instance, the BFSStorage will replace the `data`
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   430
        #    binary value with a Binary containing the destination path
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   431
        #    on the filesystem. To make the entity.data usage absolutely
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   432
        #    transparent, we'll have to reset entity.data to its binary
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   433
        #    value once the SQL query will be executed
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   434
        orig_values = {}
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   435
        etype = entity.__regid__
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   436
        for attr, storage in self._storages.get(etype, {}).items():
5011
57c4c6399b44 [source storage] on deletion, entity has no edited_attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4964
diff changeset
   437
            try:
57c4c6399b44 [source storage] on deletion, entity has no edited_attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4964
diff changeset
   438
                if attr in entity.edited_attributes:
57c4c6399b44 [source storage] on deletion, entity has no edited_attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4964
diff changeset
   439
                    orig_values[attr] = entity[attr]
57c4c6399b44 [source storage] on deletion, entity has no edited_attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4964
diff changeset
   440
                    handler = getattr(storage, 'entity_%s' % event)
57c4c6399b44 [source storage] on deletion, entity has no edited_attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4964
diff changeset
   441
                    handler(entity, attr)
57c4c6399b44 [source storage] on deletion, entity has no edited_attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4964
diff changeset
   442
            except AttributeError:
57c4c6399b44 [source storage] on deletion, entity has no edited_attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4964
diff changeset
   443
                assert event == 'deleted'
57c4c6399b44 [source storage] on deletion, entity has no edited_attributes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4964
diff changeset
   444
                getattr(storage, 'entity_deleted')(entity, attr)
4964
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   445
        yield # 2/ execute the source's instructions
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   446
        # 3/ restore original values
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   447
        for attr, value in orig_values.items():
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   448
            entity[attr] = value
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   449
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   450
    def add_entity(self, session, entity):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   451
        """add a new entity to the source"""
4964
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   452
        with self._storage_handler(entity, 'added'):
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   453
            attrs = self.preprocess_entity(entity)
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   454
            sql = self.sqlgen.insert(SQL_PREFIX + entity.__regid__, attrs)
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   455
            self.doexec(session, sql, attrs)
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   456
            if session.undoable_action('C', entity.__regid__):
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   457
                self._record_tx_action(session, 'tx_entity_actions', 'C',
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   458
                                       etype=entity.__regid__, eid=entity.eid)
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   459
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   460
    def update_entity(self, session, entity):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   461
        """replace an entity in the source"""
4964
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   462
        with self._storage_handler(entity, 'updated'):
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   463
            attrs = self.preprocess_entity(entity)
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   464
            if session.undoable_action('U', entity.__regid__):
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   465
                changes = self._save_attrs(session, entity, attrs)
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   466
                self._record_tx_action(session, 'tx_entity_actions', 'U',
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   467
                                       etype=entity.__regid__, eid=entity.eid,
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   468
                                       changes=self._binary(dumps(changes)))
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   469
            sql = self.sqlgen.update(SQL_PREFIX + entity.__regid__, attrs,
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   470
                                     ['cw_eid'])
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   471
            self.doexec(session, sql, attrs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   472
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   473
    def delete_entity(self, session, entity):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   474
        """delete an entity from the source"""
4964
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   475
        with self._storage_handler(entity, 'deleted'):
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   476
            if session.undoable_action('D', entity.__regid__):
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   477
                attrs = [SQL_PREFIX + r.type
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   478
                         for r in entity.e_schema.subject_relations()
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   479
                         if (r.final or r.inlined) and not r in VIRTUAL_RTYPES]
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   480
                changes = self._save_attrs(session, entity, attrs)
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   481
                self._record_tx_action(session, 'tx_entity_actions', 'D',
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   482
                                       etype=entity.__regid__, eid=entity.eid,
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   483
                                       changes=self._binary(dumps(changes)))
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   484
            attrs = {'cw_eid': entity.eid}
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   485
            sql = self.sqlgen.delete(SQL_PREFIX + entity.__regid__, attrs)
d9e8af8a7a42 [source] implement storages right in the source rather than in hooks
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4943
diff changeset
   486
            self.doexec(session, sql, attrs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   487
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   488
    def _add_relation(self, session, subject, rtype, object, inlined=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   489
        """add a relation to the source"""
4818
9f9bfbcdecfd le patch massiveimport a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4810
diff changeset
   490
        if inlined is False:
9f9bfbcdecfd le patch massiveimport a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4810
diff changeset
   491
            attrs = {'eid_from': subject, 'eid_to': object}
9f9bfbcdecfd le patch massiveimport a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4810
diff changeset
   492
            sql = self.sqlgen.insert('%s_relation' % rtype, attrs)
9f9bfbcdecfd le patch massiveimport a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4810
diff changeset
   493
        else: # used by data import
9f9bfbcdecfd le patch massiveimport a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4810
diff changeset
   494
            etype = session.describe(subject)[0]
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   495
            attrs = {'cw_eid': subject, SQL_PREFIX + rtype: object}
4818
9f9bfbcdecfd le patch massiveimport a été importé
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4810
diff changeset
   496
            sql = self.sqlgen.update(SQL_PREFIX + etype, attrs,
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   497
                                     ['cw_eid'])
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   498
        self.doexec(session, sql, attrs)
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   499
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   500
    def add_relation(self, session, subject, rtype, object, inlined=False):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   501
        """add a relation to the source"""
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   502
        self._add_relation(session, subject, rtype, object, inlined)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   503
        if session.undoable_action('A', rtype):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   504
            self._record_tx_action(session, 'tx_relation_actions', 'A',
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   505
                                   eid_from=subject, rtype=rtype, eid_to=object)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   506
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   507
    def delete_relation(self, session, subject, rtype, object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   508
        """delete a relation from the source"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   509
        rschema = self.schema.rschema(rtype)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   510
        if rschema.inlined:
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1079
diff changeset
   511
            table = SQL_PREFIX + session.describe(subject)[0]
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1079
diff changeset
   512
            column = SQL_PREFIX + rtype
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1079
diff changeset
   513
            sql = 'UPDATE %s SET %s=NULL WHERE %seid=%%(eid)s' % (table, column,
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 1079
diff changeset
   514
                                                                  SQL_PREFIX)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   515
            attrs = {'eid' : subject}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   516
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   517
            attrs = {'eid_from': subject, 'eid_to': object}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   518
            sql = self.sqlgen.delete('%s_relation' % rtype, attrs)
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   519
        self.doexec(session, sql, attrs)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   520
        if session.undoable_action('R', rtype):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   521
            self._record_tx_action(session, 'tx_relation_actions', 'R',
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   522
                                   eid_from=subject, rtype=rtype, eid_to=object)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   523
2618
ff9b0d5bd884 [F repo sqlite schema changes] don't rollback on potentially expected schema changes failure
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2611
diff changeset
   524
    def doexec(self, session, query, args=None, rollback=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   525
        """Execute a query.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   526
        it's a function just so that it shows up in profiling
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   527
        """
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   528
        cursor = session.pool[self.uri]
2593
16d9419a4a79 F: start to handle binary debug log level on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2589
diff changeset
   529
        if server.DEBUG & server.DBG_SQL:
2625
d6012db7b93e R [server debug] more server side debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2620
diff changeset
   530
            cnx = session.pool.connection(self.uri)
d6012db7b93e R [server debug] more server side debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2620
diff changeset
   531
            # getattr to get the actual connection if cnx is a ConnectionWrapper
d6012db7b93e R [server debug] more server side debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2620
diff changeset
   532
            # instance
d6012db7b93e R [server debug] more server side debugging tweaks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2620
diff changeset
   533
            print 'exec', query, args, getattr(cnx, '_cnx', cnx)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   534
        try:
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   535
            # str(query) to avoid error if it's an unicode string
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   536
            cursor.execute(str(query), args)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   537
        except Exception, ex:
4682
4994901b7379 don't issue critical message when trying to alter sqlite db during test
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4517
diff changeset
   538
            if self.repo.config.mode != 'test':
4994901b7379 don't issue critical message when trying to alter sqlite db during test
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4517
diff changeset
   539
                # during test we get those message when trying to alter sqlite
4994901b7379 don't issue critical message when trying to alter sqlite db during test
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4517
diff changeset
   540
                # db schema
4994901b7379 don't issue critical message when trying to alter sqlite db during test
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4517
diff changeset
   541
                self.critical("sql: %r\n args: %s\ndbms message: %r",
4994901b7379 don't issue critical message when trying to alter sqlite db during test
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4517
diff changeset
   542
                              query, args, ex.args[0])
2618
ff9b0d5bd884 [F repo sqlite schema changes] don't rollback on potentially expected schema changes failure
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2611
diff changeset
   543
            if rollback:
ff9b0d5bd884 [F repo sqlite schema changes] don't rollback on potentially expected schema changes failure
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2611
diff changeset
   544
                try:
ff9b0d5bd884 [F repo sqlite schema changes] don't rollback on potentially expected schema changes failure
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2611
diff changeset
   545
                    session.pool.connection(self.uri).rollback()
4692
11a040e2601c [test] also hide this message during test
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4682
diff changeset
   546
                    if self.repo.config.mode != 'test':
11a040e2601c [test] also hide this message during test
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4682
diff changeset
   547
                        self.critical('transaction has been rollbacked')
2618
ff9b0d5bd884 [F repo sqlite schema changes] don't rollback on potentially expected schema changes failure
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2611
diff changeset
   548
                except:
ff9b0d5bd884 [F repo sqlite schema changes] don't rollback on potentially expected schema changes failure
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2611
diff changeset
   549
                    pass
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   550
            raise
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   551
        return cursor
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   552
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   553
    def doexecmany(self, session, query, args):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   554
        """Execute a query.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   555
        it's a function just so that it shows up in profiling
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   556
        """
2593
16d9419a4a79 F: start to handle binary debug log level on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2589
diff changeset
   557
        if server.DEBUG & server.DBG_SQL:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   558
            print 'execmany', query, 'with', len(args), 'arguments'
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   559
        cursor = session.pool[self.uri]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   560
        try:
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   561
            # str(query) to avoid error if it's an unicode string
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   562
            cursor.executemany(str(query), args)
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   563
        except Exception, ex:
4682
4994901b7379 don't issue critical message when trying to alter sqlite db during test
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4517
diff changeset
   564
            if self.repo.config.mode != 'test':
4994901b7379 don't issue critical message when trying to alter sqlite db during test
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4517
diff changeset
   565
                # during test we get those message when trying to alter sqlite
4994901b7379 don't issue critical message when trying to alter sqlite db during test
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4517
diff changeset
   566
                # db schema
4994901b7379 don't issue critical message when trying to alter sqlite db during test
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4517
diff changeset
   567
                self.critical("sql many: %r\n args: %s\ndbms message: %r",
4994901b7379 don't issue critical message when trying to alter sqlite db during test
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4517
diff changeset
   568
                              query, args, ex.args[0])
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   569
            try:
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   570
                session.pool.connection(self.uri).rollback()
4692
11a040e2601c [test] also hide this message during test
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4682
diff changeset
   571
                if self.repo.config.mode != 'test':
11a040e2601c [test] also hide this message during test
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4682
diff changeset
   572
                    self.critical('transaction has been rollbacked')
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   573
            except:
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   574
                pass
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   575
            raise
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   576
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   577
    # short cut to method requiring advanced db helper usage ##################
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   578
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   579
    def binary_to_str(self, value):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   580
        return self.dbhelper.dbapi_module.binary_to_str(value)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   581
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   582
    def create_index(self, session, table, column, unique=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   583
        cursor = LogCursor(session.pool[self.uri])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   584
        self.dbhelper.create_index(cursor, table, column, unique)
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   585
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   586
    def drop_index(self, session, table, column, unique=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   587
        cursor = LogCursor(session.pool[self.uri])
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   588
        self.dbhelper.drop_index(cursor, table, column, unique)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   589
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   590
    # system source interface #################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   591
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   592
    def eid_type_source(self, session, eid):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   593
        """return a tuple (type, source, extid) for the entity with id <eid>"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   594
        sql = 'SELECT type, source, extid FROM entities WHERE eid=%s' % eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   595
        try:
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   596
            res = self.doexec(session, sql).fetchone()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   597
        except:
1079
452cb76fe07a backport typo fix
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 973
diff changeset
   598
            assert session.pool, 'session has no pool set'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   599
            raise UnknownEid(eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   600
        if res is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   601
            raise UnknownEid(eid)
1952
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
   602
        if res[-1] is not None:
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
   603
            if not isinstance(res, list):
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
   604
                res = list(res)
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
   605
            res[-1] = b64decode(res[-1])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   606
        return res
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   607
1952
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
   608
    def extid2eid(self, session, source, extid):
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
   609
        """get eid from an external id. Return None if no record found."""
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
   610
        assert isinstance(extid, str)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   611
        cursor = self.doexec(session,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   612
                             'SELECT eid FROM entities '
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   613
                             'WHERE extid=%(x)s AND source=%(s)s',
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   614
                             {'x': b64encode(extid), 's': source.uri})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   615
        # XXX testing rowcount cause strange bug with sqlite, results are there
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   616
        #     but rowcount is 0
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   617
        #if cursor.rowcount > 0:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   618
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   619
            result = cursor.fetchone()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   620
            if result:
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1952
diff changeset
   621
                return result[0]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   622
        except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   623
            pass
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   624
        return None
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   625
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   626
    def temp_table_def(self, selected, sol, table):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   627
        return make_schema(selected, sol, table, self.dbhelper.TYPE_MAPPING)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   628
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   629
    def create_temp_table(self, session, table, schema):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   630
        # we don't want on commit drop, this may cause problem when
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   631
        # running with an ldap source, and table will be deleted manually any way
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   632
        # on commit
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   633
        sql = self.dbhelper.sql_temporary_table(table, schema, False)
2306
95da5d9f0870 give session to doexec so it's able to rollback the connection on unexpected error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2072
diff changeset
   634
        self.doexec(session, sql)
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   635
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   636
    def create_eid(self, session):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   637
        self._eid_creation_lock.acquire()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   638
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   639
            for sql in self.dbhelper.sqls_increment_sequence('entities_id_seq'):
2310
80fcdfbb8eed oops, needs the cursor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2306
diff changeset
   640
                cursor = self.doexec(session, sql)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   641
            return cursor.fetchone()[0]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   642
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   643
            self._eid_creation_lock.release()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   644
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   645
    def add_info(self, session, entity, source, extid, complete):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   646
        """add type and source info for an eid into the system table"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   647
        # begin by inserting eid/type/source/extid into the entities table
1952
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
   648
        if extid is not None:
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
   649
            assert isinstance(extid, str)
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
   650
            extid = b64encode(extid)
3399
2b84f4adb6f8 use __regid__ instead of id on appobject classes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3240
diff changeset
   651
        attrs = {'type': entity.__regid__, 'eid': entity.eid, 'extid': extid,
1952
8e19c813750d fix extid handling: ensure encoded string is given, and store them as base64 (see note in native.py).
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1792
diff changeset
   652
                 'source': source.uri, 'mtime': datetime.now()}
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   653
        self.doexec(session, self.sqlgen.insert('entities', attrs), attrs)
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   654
        # now we can update the full text index
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   655
        if self.do_fti and self.need_fti_indexation(entity.__regid__):
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   656
            if complete:
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   657
                entity.complete(entity.e_schema.indexable_attributes())
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   658
            FTIndexEntityOp(session, entity=entity)
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   659
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   660
    def update_info(self, session, entity, need_fti_update):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   661
        """mark entity as being modified, fulltext reindex if needed"""
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   662
        if self.do_fti and need_fti_update:
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   663
            # reindex the entity only if this query is updating at least
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   664
            # one indexable attribute
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   665
            FTIndexEntityOp(session, entity=entity)
5059
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   666
        # update entities.mtime.
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   667
        # XXX Only if entity.__regid__ in self.multisources_etypes?
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
   668
        attrs = {'eid': entity.eid, 'mtime': datetime.now()}
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   669
        self.doexec(session, self.sqlgen.update('entities', attrs, ['eid']), attrs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   670
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   671
    def delete_info(self, session, entity, uri, extid):
5059
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   672
        """delete system information on deletion of an entity:
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   673
        * remove record from the entities table
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   674
        * transfer it to the deleted_entities table if the entity's type is
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   675
          multi-sources
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   676
        """
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   677
        attrs = {'eid': entity.eid}
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   678
        self.doexec(session, self.sqlgen.delete('entities', attrs), attrs)
5059
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   679
        if not entity.__regid__ in self.multisources_etypes:
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   680
            return
2955
6bb5025c9fc7 remove some pretty old deprecation code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2759
diff changeset
   681
        if extid is not None:
6bb5025c9fc7 remove some pretty old deprecation code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2759
diff changeset
   682
            assert isinstance(extid, str), type(extid)
6bb5025c9fc7 remove some pretty old deprecation code
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2759
diff changeset
   683
            extid = b64encode(extid)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   684
        attrs = {'type': entity.__regid__, 'eid': entity.eid, 'extid': extid,
5061
cdab5220eac0 [cleanup]
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5059
diff changeset
   685
                 'source': uri, 'dtime': datetime.now()}
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   686
        self.doexec(session, self.sqlgen.insert('deleted_entities', attrs), attrs)
1792
9eadf34fd860 delete-trailing-spaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   687
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   688
    def modified_entities(self, session, etypes, mtime):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   689
        """return a 2-uple:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   690
        * list of (etype, eid) of entities of the given types which have been
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   691
          modified since the given timestamp (actually entities whose full text
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   692
          index content has changed)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   693
        * list of (etype, eid) of entities of the given types which have been
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   694
          deleted since the given timestamp
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   695
        """
5059
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   696
        for etype in etypes:
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   697
            if not etype in self.multisources_etypes:
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   698
                self.critical('%s not listed as a multi-sources entity types. '
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   699
                              'Modify your configuration')
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   700
                self.multisources_etypes.add(etype)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   701
        modsql = _modified_sql('entities', etypes)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   702
        cursor = self.doexec(session, modsql, {'time': mtime})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   703
        modentities = cursor.fetchall()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   704
        delsql = _modified_sql('deleted_entities', etypes)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   705
        cursor = self.doexec(session, delsql, {'time': mtime})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   706
        delentities = cursor.fetchall()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   707
        return modentities, delentities
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   708
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   709
    # undo support #############################################################
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   710
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   711
    def undoable_transactions(self, session, ueid=None, **actionfilters):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   712
        """See :class:`cubicweb.dbapi.Connection.undoable_transactions`"""
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   713
        # force filtering to session's user if not a manager
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   714
        if not session.user.is_in_group('managers'):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   715
            ueid = session.user.eid
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   716
        restr = {}
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   717
        if ueid is not None:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   718
            restr['tx_user'] = ueid
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   719
        sql = self.sqlgen.select('transactions', restr, ('tx_uuid', 'tx_time', 'tx_user'))
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   720
        if actionfilters:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   721
            # we will need subqueries to filter transactions according to
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   722
            # actions done
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   723
            tearestr = {} # filters on the tx_entity_actions table
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   724
            trarestr = {} # filters on the tx_relation_actions table
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   725
            genrestr = {} # generic filters, appliyable to both table
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   726
            # unless public explicitly set to false, we only consider public
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   727
            # actions
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   728
            if actionfilters.pop('public', True):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   729
                genrestr['txa_public'] = True
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   730
            # put additional filters in trarestr and/or tearestr
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   731
            for key, val in actionfilters.iteritems():
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   732
                if key == 'etype':
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   733
                    # filtering on etype implies filtering on entity actions
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   734
                    # only, and with no eid specified
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   735
                    assert actionfilters.get('action', 'C') in 'CUD'
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   736
                    assert not 'eid' in actionfilters
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   737
                    tearestr['etype'] = val
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   738
                elif key == 'eid':
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   739
                    # eid filter may apply to 'eid' of tx_entity_actions or to
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   740
                    # 'eid_from' OR 'eid_to' of tx_relation_actions
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   741
                    if actionfilters.get('action', 'C') in 'CUD':
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   742
                        tearestr['eid'] = val
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   743
                    if actionfilters.get('action', 'A') in 'AR':
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   744
                        trarestr['eid_from'] = val
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   745
                        trarestr['eid_to'] = val
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   746
                elif key == 'action':
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   747
                    if val in 'CUD':
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   748
                        tearestr['txa_action'] = val
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   749
                    else:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   750
                        assert val in 'AR'
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   751
                        trarestr['txa_action'] = val
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   752
                else:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   753
                    raise AssertionError('unknow filter %s' % key)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   754
            assert trarestr or tearestr, "can't only filter on 'public'"
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   755
            subqsqls = []
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   756
            # append subqueries to the original query, using EXISTS()
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   757
            if trarestr or (genrestr and not tearestr):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   758
                trarestr.update(genrestr)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   759
                trasql = self.sqlgen.select('tx_relation_actions', trarestr, ('1',))
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   760
                if 'eid_from' in trarestr:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   761
                    # replace AND by OR between eid_from/eid_to restriction
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   762
                    trasql = sql_or_clauses(trasql, ['eid_from = %(eid_from)s',
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   763
                                                     'eid_to = %(eid_to)s'])
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   764
                trasql += ' AND transactions.tx_uuid=tx_relation_actions.tx_uuid'
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   765
                subqsqls.append('EXISTS(%s)' % trasql)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   766
            if tearestr or (genrestr and not trarestr):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   767
                tearestr.update(genrestr)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   768
                teasql = self.sqlgen.select('tx_entity_actions', tearestr, ('1',))
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   769
                teasql += ' AND transactions.tx_uuid=tx_entity_actions.tx_uuid'
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   770
                subqsqls.append('EXISTS(%s)' % teasql)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   771
            if restr:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   772
                sql += ' AND %s' % ' OR '.join(subqsqls)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   773
            else:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   774
                sql += ' WHERE %s' % ' OR '.join(subqsqls)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   775
            restr.update(trarestr)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   776
            restr.update(tearestr)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   777
        # we want results ordered by transaction's time descendant
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   778
        sql += ' ORDER BY tx_time DESC'
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   779
        cu = self.doexec(session, sql, restr)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   780
        # turn results into transaction objects
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   781
        return [tx.Transaction(*args) for args in cu.fetchall()]
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   782
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   783
    def tx_info(self, session, txuuid):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   784
        """See :class:`cubicweb.dbapi.Connection.transaction_info`"""
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   785
        return tx.Transaction(txuuid, *self._tx_info(session, txuuid))
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   786
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   787
    def tx_actions(self, session, txuuid, public):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   788
        """See :class:`cubicweb.dbapi.Connection.transaction_actions`"""
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   789
        self._tx_info(session, txuuid)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   790
        restr = {'tx_uuid': txuuid}
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   791
        if public:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   792
            restr['txa_public'] = True
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   793
        sql = self.sqlgen.select('tx_entity_actions', restr,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   794
                                 ('txa_action', 'txa_public', 'txa_order',
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   795
                                  'etype', 'eid', 'changes'))
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   796
        cu = self.doexec(session, sql, restr)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   797
        actions = [tx.EntityAction(a,p,o,et,e,c and loads(self.binary_to_str(c)))
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   798
                   for a,p,o,et,e,c in cu.fetchall()]
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   799
        sql = self.sqlgen.select('tx_relation_actions', restr,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   800
                                 ('txa_action', 'txa_public', 'txa_order',
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   801
                                  'rtype', 'eid_from', 'eid_to'))
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   802
        cu = self.doexec(session, sql, restr)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   803
        actions += [tx.RelationAction(*args) for args in cu.fetchall()]
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   804
        return sorted(actions, key=lambda x: x.order)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   805
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   806
    def undo_transaction(self, session, txuuid):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   807
        """See :class:`cubicweb.dbapi.Connection.undo_transaction`"""
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   808
        # set mode so pool isn't released subsquently until commit/rollback
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   809
        session.mode = 'write'
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   810
        errors = []
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   811
        with hooks_control(session, session.HOOKS_DENY_ALL, 'integrity'):
4943
7f5b83578fec disable security when undoing
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4913
diff changeset
   812
            with security_enabled(session, read=False):
7f5b83578fec disable security when undoing
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4913
diff changeset
   813
                for action in reversed(self.tx_actions(session, txuuid, False)):
7f5b83578fec disable security when undoing
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4913
diff changeset
   814
                    undomethod = getattr(self, '_undo_%s' % action.action.lower())
7f5b83578fec disable security when undoing
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4913
diff changeset
   815
                    errors += undomethod(session, action)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   816
        # remove the transactions record
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   817
        self.doexec(session,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   818
                    "DELETE FROM transactions WHERE tx_uuid='%s'" % txuuid)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   819
        return errors
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   820
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   821
    def start_undoable_transaction(self, session, uuid):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   822
        """session callback to insert a transaction record in the transactions
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   823
        table when some undoable transaction is started
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   824
        """
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   825
        ueid = session.user.eid
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   826
        attrs = {'tx_uuid': uuid, 'tx_user': ueid, 'tx_time': datetime.now()}
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   827
        self.doexec(session, self.sqlgen.insert('transactions', attrs), attrs)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   828
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   829
    def _save_attrs(self, session, entity, attrs):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   830
        """return a pickleable dictionary containing current values for given
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   831
        attributes of the entity
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   832
        """
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   833
        restr = {'cw_eid': entity.eid}
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   834
        sql = self.sqlgen.select(SQL_PREFIX + entity.__regid__, restr, attrs)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   835
        cu = self.doexec(session, sql, restr)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   836
        values = dict(zip(attrs, cu.fetchone()))
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   837
        # ensure backend specific binary are converted back to string
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   838
        eschema = entity.e_schema
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   839
        for column in attrs:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   840
            # [3:] remove 'cw_' prefix
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   841
            attr = column[3:]
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   842
            if not eschema.subjrels[attr].final:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   843
                continue
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   844
            if eschema.destination(attr) in ('Password', 'Bytes'):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   845
                value = values[column]
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   846
                if value is not None:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   847
                    values[column] = self.binary_to_str(value)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   848
        return values
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   849
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   850
    def _record_tx_action(self, session, table, action, **kwargs):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   851
        """record a transaction action in the given table (either
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   852
        'tx_entity_actions' or 'tx_relation_action')
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   853
        """
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   854
        kwargs['tx_uuid'] = session.transaction_uuid()
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   855
        kwargs['txa_action'] = action
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   856
        kwargs['txa_order'] = session.transaction_inc_action_counter()
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   857
        kwargs['txa_public'] = session.running_dbapi_query
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   858
        self.doexec(session, self.sqlgen.insert(table, kwargs), kwargs)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   859
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   860
    def _tx_info(self, session, txuuid):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   861
        """return transaction's time and user of the transaction with the given uuid.
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   862
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   863
        raise `NoSuchTransaction` if there is no such transaction of if the
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   864
        session's user isn't allowed to see it.
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   865
        """
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   866
        restr = {'tx_uuid': txuuid}
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   867
        sql = self.sqlgen.select('transactions', restr, ('tx_time', 'tx_user'))
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   868
        cu = self.doexec(session, sql, restr)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   869
        try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   870
            time, ueid = cu.fetchone()
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   871
        except TypeError:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   872
            raise tx.NoSuchTransaction()
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   873
        if not (session.user.is_in_group('managers')
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   874
                or session.user.eid == ueid):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   875
            raise tx.NoSuchTransaction()
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   876
        return time, ueid
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   877
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   878
    def _undo_d(self, session, action):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   879
        """undo an entity deletion"""
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   880
        errors = []
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   881
        err = errors.append
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   882
        eid = action.eid
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   883
        etype = action.etype
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   884
        _ = session._
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   885
        # get an entity instance
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   886
        try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   887
            entity = self.repo.vreg['etypes'].etype_class(etype)(session)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   888
        except Exception:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   889
            err("can't restore entity %s of type %s, type no more supported"
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   890
                % (eid, etype))
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   891
            return errors
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   892
        # check for schema changes, entities linked through inlined relation
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   893
        # still exists, rewrap binary values
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   894
        eschema = entity.e_schema
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   895
        getrschema = eschema.subjrels
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   896
        for column, value in action.changes.items():
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   897
            rtype = column[3:] # remove cw_ prefix
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   898
            try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   899
                rschema = getrschema[rtype]
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   900
            except KeyError:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   901
                err(_("Can't restore relation %(rtype)s of entity %(eid)s, "
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   902
                      "this relation does not exists anymore in the schema.")
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   903
                    % {'rtype': rtype, 'eid': eid})
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   904
            if not rschema.final:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   905
                assert value is None
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   906
                    # try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   907
                    #     tentity = session.entity_from_eid(eid)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   908
                    # except UnknownEid:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   909
                    #     err(_("Can't restore %(role)s relation %(rtype)s to "
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   910
                    #           "entity %(eid)s which doesn't exist anymore.")
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   911
                    #         % {'role': _('subject'),
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   912
                    #            'rtype': _(rtype),
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   913
                    #            'eid': eid})
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   914
                    #     continue
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   915
                    # rdef = rdefs[(eschema, tentity.__regid__)]
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   916
                    # try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   917
                    #     _undo_check_relation_target(tentity, rdef, 'object')
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   918
                    # except UndoException, ex:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   919
                    #     err(unicode(ex))
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   920
                    #     continue
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   921
                    # if rschema.inlined:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   922
                    #     entity[rtype] = value
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   923
                    # else:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   924
                    #     # restore relation where inlined changed since the deletion
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   925
                    #     del action.changes[column]
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   926
                    #     self._add_relation(session, subject, rtype, object)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   927
                    # # set related cache
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   928
                    # session.update_rel_cache_add(eid, rtype, value,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   929
                    #                              rschema.symmetric)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   930
            elif eschema.destination(rtype) in ('Bytes', 'Password'):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   931
                action.changes[column] = self._binary(value)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   932
                entity[rtype] = Binary(value)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   933
            elif isinstance(value, str):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   934
                entity[rtype] = unicode(value, session.encoding, 'replace')
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   935
            else:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   936
                entity[rtype] = value
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   937
        entity.set_eid(eid)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   938
        entity.edited_attributes = set(entity)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   939
        entity.check()
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   940
        self.repo.hm.call_hooks('before_add_entity', session, entity=entity)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   941
        # restore the entity
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   942
        action.changes['cw_eid'] = eid
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   943
        sql = self.sqlgen.insert(SQL_PREFIX + etype, action.changes)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   944
        self.doexec(session, sql, action.changes)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   945
        # restore record in entities (will update fti if needed)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   946
        self.add_info(session, entity, self, None, True)
5059
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   947
        # remove record from deleted_entities if entity's type is multi-sources
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   948
        if entity.__regid__ in self.multisources_etypes:
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   949
            self.doexec(session,
1d5c81588144 [repo] make etype which should go in deleted_entities configurable: we only need this for types imported from other multi-sources instances
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5013
diff changeset
   950
                        'DELETE FROM deleted_entities WHERE eid=%s' % eid)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   951
        self.repo.hm.call_hooks('after_add_entity', session, entity=entity)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   952
        return errors
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   953
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   954
    def _undo_r(self, session, action):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   955
        """undo a relation removal"""
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   956
        errors = []
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   957
        err = errors.append
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   958
        _ = session._
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   959
        subj, rtype, obj = action.eid_from, action.rtype, action.eid_to
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   960
        entities = []
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   961
        for role, eid in (('subject', subj), ('object', obj)):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   962
            try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   963
                entities.append(session.entity_from_eid(eid))
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   964
            except UnknownEid:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   965
                err(_("Can't restore relation %(rtype)s, %(role)s entity %(eid)s"
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   966
                      " doesn't exist anymore.")
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   967
                    % {'role': _(role),
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   968
                       'rtype': _(rtype),
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   969
                       'eid': eid})
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   970
        if not len(entities) == 2:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   971
            return errors
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   972
        sentity, oentity = entities
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   973
        try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   974
            rschema = self.schema.rschema(rtype)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   975
            rdef = rschema.rdefs[(sentity.__regid__, oentity.__regid__)]
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   976
        except KeyError:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   977
            err(_("Can't restore relation %(rtype)s between %(subj)s and "
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   978
                  "%(obj)s, that relation does not exists anymore in the "
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   979
                  "schema.")
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   980
                % {'rtype': rtype,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   981
                   'subj': subj,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   982
                   'obj': obj})
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   983
        else:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   984
            for role, entity in (('subject', sentity),
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   985
                                 ('object', oentity)):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   986
                try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   987
                    _undo_check_relation_target(entity, rdef, role)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   988
                except UndoException, ex:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   989
                    err(unicode(ex))
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   990
                    continue
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   991
        if not errors:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   992
            self.repo.hm.call_hooks('before_add_relation', session,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   993
                                    eidfrom=subj, rtype=rtype, eidto=obj)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   994
            # add relation in the database
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   995
            self._add_relation(session, subj, rtype, obj, rschema.inlined)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   996
            # set related cache
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   997
            session.update_rel_cache_add(subj, rtype, obj, rschema.symmetric)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   998
            self.repo.hm.call_hooks('after_add_relation', session,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
   999
                                    eidfrom=subj, rtype=rtype, eidto=obj)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1000
        return errors
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1001
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1002
    def _undo_c(self, session, action):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1003
        """undo an entity creation"""
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1004
        return ['undoing of entity creation not yet supported.']
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1005
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1006
    def _undo_u(self, session, action):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1007
        """undo an entity update"""
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1008
        return ['undoing of entity updating not yet supported.']
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1009
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1010
    def _undo_a(self, session, action):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1011
        """undo a relation addition"""
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1012
        return ['undoing of relation addition not yet supported.']
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1013
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1014
    # full text index handling #################################################
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1015
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1016
    @cached
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1017
    def need_fti_indexation(self, etype):
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1018
        eschema = self.schema.eschema(etype)
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1019
        if any(eschema.indexable_attributes()):
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1020
            return True
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1021
        if any(eschema.fulltext_containers()):
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1022
            return True
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1023
        return False
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1024
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1025
    def index_entity(self, session, entity):
4807
5642bfa43236 [cleanup] add index_entity to abstract source, add docstring
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4806
diff changeset
  1026
        """create an operation to [re]index textual content of the given entity
5642bfa43236 [cleanup] add index_entity to abstract source, add docstring
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4806
diff changeset
  1027
        on commit
5642bfa43236 [cleanup] add index_entity to abstract source, add docstring
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4806
diff changeset
  1028
        """
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1029
        FTIndexEntityOp(session, entity=entity)
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1030
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1031
    def fti_unindex_entity(self, session, eid):
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1032
        """remove text content for entity with the given eid from the full text
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1033
        index
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1034
        """
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1035
        try:
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4829
diff changeset
  1036
            self.dbhelper.cursor_unindex_object(eid, session.pool['system'])
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1037
        except Exception: # let KeyboardInterrupt / SystemExit propagate
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1038
            self.exception('error while unindexing %s', eid)
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1039
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1040
    def fti_index_entity(self, session, entity):
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1041
        """add text content of a created/modified entity to the full text index
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1042
        """
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1043
        self.debug('reindexing %r', entity.eid)
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1044
        try:
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1045
            # use cursor_index_object, not cursor_reindex_object since
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1046
            # unindexing done in the FTIndexEntityOp
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4829
diff changeset
  1047
            self.dbhelper.cursor_index_object(entity.eid, entity,
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4829
diff changeset
  1048
                                              session.pool['system'])
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1049
        except Exception: # let KeyboardInterrupt / SystemExit propagate
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1050
            self.exception('error while reindexing %s', entity)
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1051
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1052
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1053
class FTIndexEntityOp(hook.LateOperation):
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1054
    """operation to delay entity full text indexation to commit
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1055
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1056
    since fti indexing may trigger discovery of other entities, it should be
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1057
    triggered on precommit, not commit, and this should be done after other
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1058
    precommit operation which may add relations to the entity
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1059
    """
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1060
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1061
    def precommit_event(self):
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1062
        session = self.session
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1063
        entity = self.entity
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1064
        if entity.eid in session.transaction_data.get('pendingeids', ()):
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1065
            return # entity added and deleted in the same transaction
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1066
        alreadydone = session.transaction_data.setdefault('indexedeids', set())
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1067
        if entity.eid in alreadydone:
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1068
            self.debug('skipping reindexation of %s, already done', entity.eid)
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1069
            return
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1070
        alreadydone.add(entity.eid)
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1071
        source = session.repo.system_source
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1072
        for container in entity.fti_containers():
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1073
            source.fti_unindex_entity(session, container.eid)
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1074
            source.fti_index_entity(session, container)
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1075
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1076
    def commit_event(self):
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1077
        pass
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4692
diff changeset
  1078
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1079
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1080
def sql_schema(driver):
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4829
diff changeset
  1081
    helper = get_db_helper(driver)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1082
    typemap = helper.TYPE_MAPPING
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1083
    schema = """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1084
/* Create the repository's system database */
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1085
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1086
%s
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1087
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1088
CREATE TABLE entities (
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1089
  eid INTEGER PRIMARY KEY NOT NULL,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1090
  type VARCHAR(64) NOT NULL,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1091
  source VARCHAR(64) NOT NULL,
4113
986fc01be83c TIMESTAMP column type has a special meaning for SQLServer,
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 3958
diff changeset
  1092
  mtime %s NOT NULL,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1093
  extid VARCHAR(256)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1094
);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1095
CREATE INDEX entities_type_idx ON entities(type);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1096
CREATE INDEX entities_mtime_idx ON entities(mtime);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1097
CREATE INDEX entities_extid_idx ON entities(extid);;
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1098
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1099
CREATE TABLE deleted_entities (
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1100
  eid INTEGER PRIMARY KEY NOT NULL,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1101
  type VARCHAR(64) NOT NULL,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1102
  source VARCHAR(64) NOT NULL,
4113
986fc01be83c TIMESTAMP column type has a special meaning for SQLServer,
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 3958
diff changeset
  1103
  dtime %s NOT NULL,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1104
  extid VARCHAR(256)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1105
);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1106
CREATE INDEX deleted_entities_type_idx ON deleted_entities(type);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1107
CREATE INDEX deleted_entities_dtime_idx ON deleted_entities(dtime);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1108
CREATE INDEX deleted_entities_extid_idx ON deleted_entities(extid);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1109
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1110
CREATE TABLE transactions (
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1111
  tx_uuid CHAR(32) PRIMARY KEY NOT NULL,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1112
  tx_user INTEGER NOT NULL,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1113
  tx_time %s NOT NULL
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1114
);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1115
CREATE INDEX transactions_tx_user_idx ON transactions(tx_user);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1116
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1117
CREATE TABLE tx_entity_actions (
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1118
  tx_uuid CHAR(32) REFERENCES transactions(tx_uuid) ON DELETE CASCADE,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1119
  txa_action CHAR(1) NOT NULL,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1120
  txa_public %s NOT NULL,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1121
  txa_order INTEGER,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1122
  eid INTEGER NOT NULL,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1123
  etype VARCHAR(64) NOT NULL,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1124
  changes %s
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1125
);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1126
CREATE INDEX tx_entity_actions_txa_action_idx ON tx_entity_actions(txa_action);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1127
CREATE INDEX tx_entity_actions_txa_public_idx ON tx_entity_actions(txa_public);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1128
CREATE INDEX tx_entity_actions_eid_idx ON tx_entity_actions(eid);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1129
CREATE INDEX tx_entity_actions_etype_idx ON tx_entity_actions(etype);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1130
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1131
CREATE TABLE tx_relation_actions (
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1132
  tx_uuid CHAR(32) REFERENCES transactions(tx_uuid) ON DELETE CASCADE,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1133
  txa_action CHAR(1) NOT NULL,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1134
  txa_public %s NOT NULL,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1135
  txa_order INTEGER,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1136
  eid_from INTEGER NOT NULL,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1137
  eid_to INTEGER NOT NULL,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1138
  rtype VARCHAR(256) NOT NULL
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1139
);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1140
CREATE INDEX tx_relation_actions_txa_action_idx ON tx_relation_actions(txa_action);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1141
CREATE INDEX tx_relation_actions_txa_public_idx ON tx_relation_actions(txa_public);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1142
CREATE INDEX tx_relation_actions_eid_from_idx ON tx_relation_actions(eid_from);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1143
CREATE INDEX tx_relation_actions_eid_to_idx ON tx_relation_actions(eid_to);;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1144
""" % (helper.sql_create_sequence('entities_id_seq').replace(';', ';;'),
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1145
       typemap['Datetime'], typemap['Datetime'], typemap['Datetime'],
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1146
       typemap['Boolean'], typemap['Bytes'], typemap['Boolean'])
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1147
    if helper.backend_name == 'sqlite':
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1148
        # sqlite support the ON DELETE CASCADE syntax but do nothing
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1149
        schema += '''
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1150
CREATE TRIGGER fkd_transactions
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1151
BEFORE DELETE ON transactions
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1152
FOR EACH ROW BEGIN
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1153
    DELETE FROM tx_entity_actions WHERE tx_uuid=OLD.tx_uuid;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1154
    DELETE FROM tx_relation_actions WHERE tx_uuid=OLD.tx_uuid;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1155
END;;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1156
'''
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1157
    return schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1158
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1159
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1160
def sql_drop_schema(driver):
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4829
diff changeset
  1161
    helper = get_db_helper(driver)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1162
    return """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1163
%s
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1164
DROP TABLE entities;
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1165
DROP TABLE deleted_entities;
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1166
DROP TABLE transactions;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1167
DROP TABLE tx_entity_actions;
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1168
DROP TABLE tx_relation_actions;
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1169
""" % helper.sql_drop_sequence('entities_id_seq')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1170
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1171
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1172
def grant_schema(user, set_owner=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1173
    result = ''
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1174
    for table in ('entities', 'deleted_entities', 'entities_id_seq',
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1175
                  'transactions', 'tx_entity_actions', 'tx_relation_actions'):
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1176
        if set_owner:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1177
            result = 'ALTER TABLE %s OWNER TO %s;\n' % (table, user)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4902
diff changeset
  1178
        result += 'GRANT ALL ON %s TO %s;\n' % (table, user)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1179
    return result
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1180
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1181
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1182
class BaseAuthentifier(object):
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1183
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1184
    def __init__(self, source=None):
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1185
        self.source = source
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1186
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1187
    def set_schema(self, schema):
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1188
        """set the instance'schema"""
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1189
        pass
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1190
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1191
class LoginPasswordAuthentifier(BaseAuthentifier):
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1192
    passwd_rql = "Any P WHERE X is CWUser, X login %(login)s, X upassword P"
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1193
    auth_rql = "Any X WHERE X is CWUser, X login %(login)s, X upassword %(pwd)s"
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1194
    _sols = ({'X': 'CWUser', 'P': 'Password'},)
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1195
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1196
    def set_schema(self, schema):
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1197
        """set the instance'schema"""
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1198
        if 'CWUser' in schema: # probably an empty schema if not true...
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1199
            # rql syntax trees used to authenticate users
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1200
            self._passwd_rqlst = self.source.compile_rql(self.passwd_rql, self._sols)
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1201
            self._auth_rqlst = self.source.compile_rql(self.auth_rql, self._sols)
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1202
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1203
    def authenticate(self, session, login, password=None, **kwargs):
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1204
        """return CWUser eid for the given login/password if this account is
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1205
        defined in this source, else raise `AuthenticationError`
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1206
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1207
        two queries are needed since passwords are stored crypted, so we have
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1208
        to fetch the salt first
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1209
        """
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1210
        args = {'login': login, 'pwd' : password}
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1211
        if password is not None:
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1212
            rset = self.source.syntax_tree_search(session, self._passwd_rqlst, args)
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1213
            try:
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1214
                pwd = rset[0][0]
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1215
            except IndexError:
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1216
                raise AuthenticationError('bad login')
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1217
            # passwords are stored using the Bytes type, so we get a StringIO
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1218
            if pwd is not None:
4204
60256056bda6 backport stable branch
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4181 4198
diff changeset
  1219
                args['pwd'] = Binary(crypt_password(password, pwd.getvalue()[:2]))
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1220
        # get eid from login and (crypted) password
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1221
        rset = self.source.syntax_tree_search(session, self._auth_rqlst, args)
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1222
        try:
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1223
            return rset[0][0]
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1224
        except IndexError:
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3503
diff changeset
  1225
            raise AuthenticationError('bad password')