server/sqlutils.py
author Denis Laxalde <denis.laxalde@logilab.fr>
Thu, 19 May 2016 10:51:33 +0200
changeset 11260 0119ba327117
parent 11035 0fb100e8385b
permissions -rw-r--r--
[pkg] Set version to 3.22.2.dev0 So that cubes used in test dependencies do not install a released CubicWeb.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
11030
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
     1
# copyright 2003-2015 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5214
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5214
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5214
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5214
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5214
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5214
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5214
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5214
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5214
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5214
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5214
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5214
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5214
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5214
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5214
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
5768
1e73a466aa69 [fti] support for fti ranking: has_text query results sorted by relevance, and provides a way to control weight per entity / entity's attribute
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5625
diff changeset
    18
"""SQL utilities functions and classes."""
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10588
diff changeset
    19
from __future__ import print_function
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
9466
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
    23
import sys
8755
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    24
import re
4298
2ca56131079e enable cubicweb-ctl db-dump and db-restore on Windows with SQL Server
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 4212
diff changeset
    25
import subprocess
9466
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
    26
from os.path import abspath
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
    27
from logging import getLogger
11030
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
    28
from datetime import time, datetime, timedelta
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 0
diff changeset
    29
10760
c34bbdb18745 [server] unicode → six.text_type
Julien Cristau <julien.cristau@logilab.fr>
parents: 10662
diff changeset
    30
from six import string_types, text_type
10607
6519ae8cca0c [py3k] import filter using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    31
from six.moves import filter
6519ae8cca0c [py3k] import filter using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    32
11034
75d752e6daf7 [server] improve TZDatetime support
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11032
diff changeset
    33
from pytz import utc
75d752e6daf7 [server] improve TZDatetime support
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11032
diff changeset
    34
4849
3827b9ee77ac missing rename
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4848
diff changeset
    35
from logilab import database as db, common as lgc
10375
28ec01db78b3 [server] Do not use progress bar when stdout is not a TTY
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 10125
diff changeset
    36
from logilab.common.shellutils import ProgressBar, DummyProgressBar
9466
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
    37
from logilab.common.deprecation import deprecated
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
    38
from logilab.common.logging_ext import set_log_methods
11030
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
    39
from logilab.common.date import utctime, utcdatetime, strptime
4848
41f84eea63c9 rename logilab.db into logilab.database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4831
diff changeset
    40
from logilab.database.sqlgen import SQLGenerator
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    41
9466
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
    42
from cubicweb import Binary, ConfigurationError
4023
eae23c40627a drop common subpackage
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
    43
from cubicweb.uilib import remove_html_tags
2596
d02eed70937f [R repo, schema] use VIRTUAL_RTYPES const
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2512
diff changeset
    44
from cubicweb.schema import PURE_VIRTUAL_RTYPES
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    45
from cubicweb.server import SQL_CONNECT_HOOKS
1132
96752791c2b6 pylint cleanup
sylvain.thenault@logilab.fr
parents: 1026
diff changeset
    46
from cubicweb.server.utils import crypt_password
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    47
1783
b81f9761907c mx flag has moved
sylvain.thenault@logilab.fr
parents: 1619
diff changeset
    48
lgc.USE_MX_DATETIME = False
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 0
diff changeset
    49
SQL_PREFIX = 'cw_'
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
11035
0fb100e8385b pep8 bits
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11034
diff changeset
    51
4353
7db69db4913c command may now officially be either a string or a list, don't make think it's for backward compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4342
diff changeset
    52
def _run_command(cmd):
10824
fbefdaa56d2b [server] fix db-dump with sqlite
Julien Cristau <julien.cristau@logilab.fr>
parents: 10760
diff changeset
    53
    if isinstance(cmd, string_types):
fbefdaa56d2b [server] fix db-dump with sqlite
Julien Cristau <julien.cristau@logilab.fr>
parents: 10760
diff changeset
    54
        print(cmd)
fbefdaa56d2b [server] fix db-dump with sqlite
Julien Cristau <julien.cristau@logilab.fr>
parents: 10760
diff changeset
    55
        return subprocess.call(cmd, shell=True)
fbefdaa56d2b [server] fix db-dump with sqlite
Julien Cristau <julien.cristau@logilab.fr>
parents: 10760
diff changeset
    56
    else:
fbefdaa56d2b [server] fix db-dump with sqlite
Julien Cristau <julien.cristau@logilab.fr>
parents: 10760
diff changeset
    57
        print(' '.join(cmd))
fbefdaa56d2b [server] fix db-dump with sqlite
Julien Cristau <julien.cristau@logilab.fr>
parents: 10760
diff changeset
    58
        return subprocess.call(cmd)
4342
b4e186da08f2 handle lgc.adbh api changes within regards of backup/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4322
diff changeset
    59
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    60
9891
3386fd89c914 remove references to global environment variable APYCOT_ROOT
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9724
diff changeset
    61
def sqlexec(sqlstmts, cursor_or_execute, withpb=True,
8755
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    62
            pbtitle='', delimiter=';', cnx=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
    """execute sql statements ignoring DROP/ CREATE GROUP or USER statements
8755
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    64
    error.
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    65
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    66
    :sqlstmts_as_string: a string or a list of sql statements.
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    67
    :cursor_or_execute: sql cursor or a callback used to execute statements
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    68
    :cnx: if given, commit/rollback at each statement.
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    69
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    70
    :withpb: if True, display a progresse bar
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    71
    :pbtitle: a string displayed as the progress bar title (if `withpb=True`)
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    72
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    73
    :delimiter: a string used to split sqlstmts (if it is a string)
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    74
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    75
    Return the failed statements (same type as sqlstmts)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    76
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    77
    if hasattr(cursor_or_execute, 'execute'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    78
        execute = cursor_or_execute.execute
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    79
    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    80
        execute = cursor_or_execute
8755
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    81
    sqlstmts_as_string = False
10612
84468b90e9c1 [py3k] basestring → six.string_types
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10607
diff changeset
    82
    if isinstance(sqlstmts, string_types):
8755
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    83
        sqlstmts_as_string = True
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    84
        sqlstmts = sqlstmts.split(delimiter)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    85
    if withpb:
10375
28ec01db78b3 [server] Do not use progress bar when stdout is not a TTY
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 10125
diff changeset
    86
        if sys.stdout.isatty():
28ec01db78b3 [server] Do not use progress bar when stdout is not a TTY
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 10125
diff changeset
    87
            pb = ProgressBar(len(sqlstmts), title=pbtitle)
28ec01db78b3 [server] Do not use progress bar when stdout is not a TTY
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 10125
diff changeset
    88
        else:
28ec01db78b3 [server] Do not use progress bar when stdout is not a TTY
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 10125
diff changeset
    89
            pb = DummyProgressBar()
8755
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    90
    failed = []
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
    for sql in sqlstmts:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
        sql = sql.strip()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
        if withpb:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    94
            pb.update()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
        if not sql:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
            continue
8755
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    97
        try:
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    98
            # some dbapi modules doesn't accept unicode for sql string
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
    99
            execute(str(sql))
11035
0fb100e8385b pep8 bits
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11034
diff changeset
   100
        except Exception:
8755
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   101
            if cnx:
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   102
                cnx.rollback()
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   103
            failed.append(sql)
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   104
        else:
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   105
            if cnx:
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   106
                cnx.commit()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   107
    if withpb:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10588
diff changeset
   108
        print()
8755
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   109
    if sqlstmts_as_string:
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   110
        failed = delimiter.join(failed)
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   111
    return failed
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   112
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   113
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   114
def sqlgrants(schema, driver, user,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   115
              text_index=True, set_owner=True,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   116
              skip_relations=(), skip_entities=()):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   117
    """return sql to give all access privileges to the given user on the system
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   118
    schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   119
    """
10200
cceb2c7c02f4 Use our version of schema2sql
Julien Cristau <julien.cristau@logilab.fr>
parents: 10125
diff changeset
   120
    from cubicweb.server.schema2sql import grant_schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   121
    from cubicweb.server.sources import native
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   122
    output = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   123
    w = output.append
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   124
    w(native.grant_schema(user, set_owner))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   125
    w('')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   126
    if text_index:
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   127
        dbhelper = db.get_db_helper(driver)
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   128
        w(dbhelper.sql_grant_user_on_fti(user))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
        w('')
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   130
    w(grant_schema(schema, user, set_owner, skip_entities=skip_entities, prefix=SQL_PREFIX))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   131
    return '\n'.join(output)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   132
1619
e4845b54a704 force proper date/datetime according to type (necessary for sqlite at least)
sylvain.thenault@logilab.fr
parents: 1408
diff changeset
   133
e4845b54a704 force proper date/datetime according to type (necessary for sqlite at least)
sylvain.thenault@logilab.fr
parents: 1408
diff changeset
   134
def sqlschema(schema, driver, text_index=True,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
              user=None, set_owner=False,
2596
d02eed70937f [R repo, schema] use VIRTUAL_RTYPES const
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2512
diff changeset
   136
              skip_relations=PURE_VIRTUAL_RTYPES, skip_entities=()):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
    """return the system sql schema, according to the given parameters"""
10200
cceb2c7c02f4 Use our version of schema2sql
Julien Cristau <julien.cristau@logilab.fr>
parents: 10125
diff changeset
   138
    from cubicweb.server.schema2sql import schema2sql
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   139
    from cubicweb.server.sources import native
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   140
    if set_owner:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
        assert user, 'user is argument required when set_owner is true'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
    output = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   143
    w = output.append
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   144
    w(native.sql_schema(driver))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   145
    w('')
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   146
    dbhelper = db.get_db_helper(driver)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   147
    if text_index:
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   148
        w(dbhelper.sql_init_fti().replace(';', ';;'))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   149
        w('')
1619
e4845b54a704 force proper date/datetime according to type (necessary for sqlite at least)
sylvain.thenault@logilab.fr
parents: 1408
diff changeset
   150
    w(schema2sql(dbhelper, schema, prefix=SQL_PREFIX,
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   151
                 skip_entities=skip_entities,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   152
                 skip_relations=skip_relations).replace(';', ';;'))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   153
    if dbhelper.users_support and user:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   154
        w('')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   155
        w(sqlgrants(schema, driver, user, text_index, set_owner,
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   156
                    skip_relations, skip_entities).replace(';', ';;'))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   157
    return '\n'.join(output)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   158
1619
e4845b54a704 force proper date/datetime according to type (necessary for sqlite at least)
sylvain.thenault@logilab.fr
parents: 1408
diff changeset
   159
e4845b54a704 force proper date/datetime according to type (necessary for sqlite at least)
sylvain.thenault@logilab.fr
parents: 1408
diff changeset
   160
def sqldropschema(schema, driver, text_index=True,
2596
d02eed70937f [R repo, schema] use VIRTUAL_RTYPES const
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2512
diff changeset
   161
                  skip_relations=PURE_VIRTUAL_RTYPES, skip_entities=()):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   162
    """return the sql to drop the schema, according to the given parameters"""
10200
cceb2c7c02f4 Use our version of schema2sql
Julien Cristau <julien.cristau@logilab.fr>
parents: 10125
diff changeset
   163
    from cubicweb.server.schema2sql import dropschema2sql
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   164
    from cubicweb.server.sources import native
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   165
    output = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   166
    w = output.append
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   167
    if text_index:
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   168
        dbhelper = db.get_db_helper(driver)
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   169
        w(dbhelper.sql_drop_fti())
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   170
        w('')
7906
203d574c8a1d repaire cctl db-init -d on sqlserver (closes #1979670)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7341
diff changeset
   171
    w(dropschema2sql(dbhelper, schema, prefix=SQL_PREFIX,
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1953
diff changeset
   172
                     skip_entities=skip_entities,
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1953
diff changeset
   173
                     skip_relations=skip_relations))
5214
3285b6e3b930 fix cwctl db-init -d on SQL Server
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5013
diff changeset
   174
    w('')
3285b6e3b930 fix cwctl db-init -d on SQL Server
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5013
diff changeset
   175
    w(native.sql_drop_schema(driver))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   176
    return '\n'.join(output)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   177
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   178
8755
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   179
_SQL_DROP_ALL_USER_TABLES_FILTER_FUNCTION = re.compile('^(?!(sql|pg)_)').match
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   180
def sql_drop_all_user_tables(driver_or_helper, sqlcursor):
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   181
    """Return ths sql to drop all tables found in the database system."""
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   182
    if not getattr(driver_or_helper, 'list_tables', None):
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   183
        dbhelper = db.get_db_helper(driver_or_helper)
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   184
    else:
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   185
        dbhelper = driver_or_helper
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   186
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   187
    cmds = [dbhelper.sql_drop_sequence('entities_id_seq')]
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   188
    # for mssql, we need to drop views before tables
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   189
    if hasattr(dbhelper, 'list_views'):
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   190
        cmds += ['DROP VIEW %s;' % name
10607
6519ae8cca0c [py3k] import filter using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   191
                 for name in filter(_SQL_DROP_ALL_USER_TABLES_FILTER_FUNCTION, dbhelper.list_views(sqlcursor))]
8755
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   192
    cmds += ['DROP TABLE %s;' % name
10607
6519ae8cca0c [py3k] import filter using six.moves
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   193
             for name in filter(_SQL_DROP_ALL_USER_TABLES_FILTER_FUNCTION, dbhelper.list_tables(sqlcursor))]
8755
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   194
    return '\n'.join(cmds)
1f3757ef3762 [server] *init_repository* lookup the database instead of the schema to drop tables (closes #810743)
Alain Leufroy <alain.leufroy@logilab.fr>
parents: 8139
diff changeset
   195
9447
0636c4960259 [multi-sources-removal] Drop cw.server.sources.extlite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9364
diff changeset
   196
9466
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   197
class ConnectionWrapper(object):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   198
    """handle connection to the system source, at some point associated to a
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   199
    :class:`Session`
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   200
    """
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   201
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   202
    # since 3.19, we only have to manage the system source connection
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   203
    def __init__(self, system_source):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   204
        # dictionary of (source, connection), indexed by sources'uri
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   205
        self._source = system_source
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   206
        self.cnx = system_source.get_connection()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   207
        self.cu = self.cnx.cursor()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   208
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   209
    def commit(self):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   210
        """commit the current transaction for this user"""
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   211
        # let exception propagates
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   212
        self.cnx.commit()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   213
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   214
    def rollback(self):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   215
        """rollback the current transaction for this user"""
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   216
        # catch exceptions, rollback other sources anyway
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   217
        try:
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   218
            self.cnx.rollback()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   219
        except Exception:
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   220
            self._source.critical('rollback error', exc_info=sys.exc_info())
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   221
            # error on rollback, the connection is much probably in a really
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   222
            # bad state. Replace it by a new one.
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   223
            self.reconnect()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   224
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   225
    def close(self, i_know_what_i_do=False):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   226
        """close all connections in the set"""
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   227
        if i_know_what_i_do is not True: # unexpected closing safety belt
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   228
            raise RuntimeError('connections set shouldn\'t be closed')
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   229
        try:
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   230
            self.cu.close()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   231
            self.cu = None
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   232
        except Exception:
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   233
            pass
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   234
        try:
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   235
            self.cnx.close()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   236
            self.cnx = None
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   237
        except Exception:
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   238
            pass
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   239
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   240
    # internals ###############################################################
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   241
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   242
    def cnxset_freed(self):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   243
        """connections set is being freed from a session"""
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   244
        pass # no nothing by default
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   245
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   246
    def reconnect(self):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   247
        """reopen a connection for this source or all sources if none specified
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   248
        """
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   249
        try:
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   250
            # properly close existing connection if any
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   251
            self.cnx.close()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   252
        except Exception:
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   253
            pass
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   254
        self._source.info('trying to reconnect')
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   255
        self.cnx = self._source.get_connection()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   256
        self.cu = self.cnx.cursor()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   257
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   258
    @deprecated('[3.19] use .cu instead')
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   259
    def __getitem__(self, uri):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   260
        assert uri == 'system'
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   261
        return self.cu
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   262
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   263
    @deprecated('[3.19] use repo.system_source instead')
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   264
    def source(self, uid):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   265
        assert uid == 'system'
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   266
        return self._source
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   267
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   268
    @deprecated('[3.19] use .cnx instead')
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   269
    def connection(self, uid):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   270
        assert uid == 'system'
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   271
        return self.cnx
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   272
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   273
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   274
class SqliteConnectionWrapper(ConnectionWrapper):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   275
    """Sqlite specific connection wrapper: close the connection each time it's
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   276
    freed (and reopen it later when needed)
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   277
    """
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   278
    def __init__(self, system_source):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   279
        # don't call parent's __init__, we don't want to initiate the connection
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   280
        self._source = system_source
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   281
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   282
    _cnx = None
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   283
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   284
    def cnxset_freed(self):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   285
        self.cu.close()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   286
        self.cnx.close()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   287
        self.cnx = self.cu = None
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   288
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   289
    @property
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   290
    def cnx(self):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   291
        if self._cnx is None:
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   292
            self._cnx = self._source.get_connection()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   293
            self._cu = self._cnx.cursor()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   294
        return self._cnx
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   295
    @cnx.setter
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   296
    def cnx(self, value):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   297
        self._cnx = value
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   298
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   299
    @property
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   300
    def cu(self):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   301
        if self._cnx is None:
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   302
            self._cnx = self._source.get_connection()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   303
            self._cu = self._cnx.cursor()
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   304
        return self._cu
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   305
    @cu.setter
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   306
    def cu(self, value):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   307
        self._cu = value
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   308
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   309
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   310
class SQLAdapterMixIn(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   311
    """Mixin for SQL data sources, getting a connection from a configuration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   312
    dictionary and handling connection locking
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   313
    """
9466
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   314
    cnx_wrap = ConnectionWrapper
1619
e4845b54a704 force proper date/datetime according to type (necessary for sqlite at least)
sylvain.thenault@logilab.fr
parents: 1408
diff changeset
   315
9724
e45bf9baa7b7 Add a db-statement-timeout option for postgresql sources
Julien Cristau <julien.cristau@logilab.fr>
parents: 9466
diff changeset
   316
    def __init__(self, source_config, repairing=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   317
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   318
            self.dbdriver = source_config['db-driver'].lower()
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   319
            dbname = source_config['db-name']
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   320
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
            raise ConfigurationError('missing some expected entries in sources file')
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   322
        dbhost = source_config.get('db-host')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
        port = source_config.get('db-port')
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   324
        dbport = port and int(port) or None
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   325
        dbuser = source_config.get('db-user')
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   326
        dbpassword = source_config.get('db-password')
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   327
        dbencoding = source_config.get('db-encoding', 'UTF-8')
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   328
        dbextraargs = source_config.get('db-extra-arguments')
10125
bc6461a7d2da [server] add a db-namespace option in source definition (closes #1631339)
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 10086
diff changeset
   329
        dbnamespace = source_config.get('db-namespace')
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   330
        self.dbhelper = db.get_db_helper(self.dbdriver)
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   331
        self.dbhelper.record_connection_info(dbname, dbhost, dbport, dbuser,
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   332
                                             dbpassword, dbextraargs,
10125
bc6461a7d2da [server] add a db-namespace option in source definition (closes #1631339)
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 10086
diff changeset
   333
                                             dbencoding, dbnamespace)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   334
        self.sqlgen = SQLGenerator()
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   335
        # copy back some commonly accessed attributes
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   336
        dbapi_module = self.dbhelper.dbapi_module
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   337
        self.OperationalError = dbapi_module.OperationalError
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   338
        self.InterfaceError = dbapi_module.InterfaceError
5605
2604545d7dd9 [win32 SQLServer] connection lost detection
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5424
diff changeset
   339
        self.DbapiError = dbapi_module.Error
6379
3f67f7ea5632 [R] use dbhelper.binary_value to process passwords and other Bytes fields
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6333
diff changeset
   340
        self._binary = self.dbhelper.binary_value
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   341
        self._process_value = dbapi_module.process_value
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   342
        self._dbencoding = dbencoding
9466
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   343
        if self.dbdriver == 'sqlite':
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   344
            self.cnx_wrap = SqliteConnectionWrapper
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   345
            self.dbhelper.dbname = abspath(self.dbhelper.dbname)
9724
e45bf9baa7b7 Add a db-statement-timeout option for postgresql sources
Julien Cristau <julien.cristau@logilab.fr>
parents: 9466
diff changeset
   346
        if not repairing:
e45bf9baa7b7 Add a db-statement-timeout option for postgresql sources
Julien Cristau <julien.cristau@logilab.fr>
parents: 9466
diff changeset
   347
            statement_timeout = int(source_config.get('db-statement-timeout', 0))
e45bf9baa7b7 Add a db-statement-timeout option for postgresql sources
Julien Cristau <julien.cristau@logilab.fr>
parents: 9466
diff changeset
   348
            if statement_timeout > 0:
e45bf9baa7b7 Add a db-statement-timeout option for postgresql sources
Julien Cristau <julien.cristau@logilab.fr>
parents: 9466
diff changeset
   349
                def set_postgres_timeout(cnx):
e45bf9baa7b7 Add a db-statement-timeout option for postgresql sources
Julien Cristau <julien.cristau@logilab.fr>
parents: 9466
diff changeset
   350
                    cnx.cursor().execute('SET statement_timeout to %d' % statement_timeout)
e45bf9baa7b7 Add a db-statement-timeout option for postgresql sources
Julien Cristau <julien.cristau@logilab.fr>
parents: 9466
diff changeset
   351
                    cnx.commit()
e45bf9baa7b7 Add a db-statement-timeout option for postgresql sources
Julien Cristau <julien.cristau@logilab.fr>
parents: 9466
diff changeset
   352
                postgres_hooks = SQL_CONNECT_HOOKS['postgres']
e45bf9baa7b7 Add a db-statement-timeout option for postgresql sources
Julien Cristau <julien.cristau@logilab.fr>
parents: 9466
diff changeset
   353
                postgres_hooks.append(set_postgres_timeout)
9466
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   354
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   355
    def wrapped_connection(self):
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   356
        """open and return a connection to the database, wrapped into a class
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   357
        handling reconnection and all
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   358
        """
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   359
        return self.cnx_wrap(self)
1619
e4845b54a704 force proper date/datetime according to type (necessary for sqlite at least)
sylvain.thenault@logilab.fr
parents: 1408
diff changeset
   360
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   361
    def get_connection(self):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   362
        """open and return a connection to the database"""
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   363
        return self.dbhelper.get_connection()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   364
4893
15ae9a33a7f2 [db backup] fix name error in backup_to_file: we've to pass .confirm all along the chain as for restore
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4854
diff changeset
   365
    def backup_to_file(self, backupfile, confirm):
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   366
        for cmd in self.dbhelper.backup_commands(backupfile,
4342
b4e186da08f2 handle lgc.adbh api changes within regards of backup/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4322
diff changeset
   367
                                                 keepownership=False):
4353
7db69db4913c command may now officially be either a string or a list, don't make think it's for backward compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4342
diff changeset
   368
            if _run_command(cmd):
7db69db4913c command may now officially be either a string or a list, don't make think it's for backward compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4342
diff changeset
   369
                if not confirm('   [Failed] Continue anyway?', default='n'):
4342
b4e186da08f2 handle lgc.adbh api changes within regards of backup/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4322
diff changeset
   370
                    raise Exception('Failed command: %s' % cmd)
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2396
diff changeset
   371
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2396
diff changeset
   372
    def restore_from_file(self, backupfile, confirm, drop=True):
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   373
        for cmd in self.dbhelper.restore_commands(backupfile,
4854
b06d2a3b27d9 logilab.db compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   374
                                                  keepownership=False,
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2396
diff changeset
   375
                                                  drop=drop):
4353
7db69db4913c command may now officially be either a string or a list, don't make think it's for backward compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4342
diff changeset
   376
            if _run_command(cmd):
7db69db4913c command may now officially be either a string or a list, don't make think it's for backward compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4342
diff changeset
   377
                if not confirm('   [Failed] Continue anyway?', default='n'):
4195
86dcaf6bb92f closes #601987
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4179
diff changeset
   378
                    raise Exception('Failed command: %s' % cmd)
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2396
diff changeset
   379
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   380
    def merge_args(self, args, query_args):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   381
        if args is not None:
4474
55fe19813bb7 kill mx compat code (dropped since 3.2), more efficient merge_args implementation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4466
diff changeset
   382
            newargs = {}
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10651
diff changeset
   383
            for key, val in args.items():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   384
                # convert cubicweb binary into db binary
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
                if isinstance(val, Binary):
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   386
                    val = self._binary(val.getvalue())
10643
cfded6d0da11 fix bad-caching of datetime with tz info at sql generation time
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10411
diff changeset
   387
                # convert timestamp to utc.
cfded6d0da11 fix bad-caching of datetime with tz info at sql generation time
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10411
diff changeset
   388
                # expect SET TiME ZONE to UTC at connection opening time.
cfded6d0da11 fix bad-caching of datetime with tz info at sql generation time
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10411
diff changeset
   389
                # This shouldn't change anything for datetime without TZ.
cfded6d0da11 fix bad-caching of datetime with tz info at sql generation time
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10411
diff changeset
   390
                elif isinstance(val, datetime) and val.tzinfo is not None:
cfded6d0da11 fix bad-caching of datetime with tz info at sql generation time
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10411
diff changeset
   391
                    val = utcdatetime(val)
cfded6d0da11 fix bad-caching of datetime with tz info at sql generation time
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10411
diff changeset
   392
                elif isinstance(val, time) and val.tzinfo is not None:
cfded6d0da11 fix bad-caching of datetime with tz info at sql generation time
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10411
diff changeset
   393
                    val = utctime(val)
4474
55fe19813bb7 kill mx compat code (dropped since 3.2), more efficient merge_args implementation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4466
diff changeset
   394
                newargs[key] = val
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   395
            # should not collide
10643
cfded6d0da11 fix bad-caching of datetime with tz info at sql generation time
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10411
diff changeset
   396
            assert not (frozenset(newargs) & frozenset(query_args)), \
cfded6d0da11 fix bad-caching of datetime with tz info at sql generation time
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10411
diff changeset
   397
                'unexpected collision: %s' % (frozenset(newargs) & frozenset(query_args))
4474
55fe19813bb7 kill mx compat code (dropped since 3.2), more efficient merge_args implementation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4466
diff changeset
   398
            newargs.update(query_args)
55fe19813bb7 kill mx compat code (dropped since 3.2), more efficient merge_args implementation
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4466
diff changeset
   399
            return newargs
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   400
        return query_args
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
10086
98bc2ca1a816 [source/native] session -> cnx
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9891
diff changeset
   402
    def process_result(self, cursor, cnx=None, column_callbacks=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   403
        """return a list of CubicWeb compliant values from data in the given cursor
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   404
        """
10086
98bc2ca1a816 [source/native] session -> cnx
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9891
diff changeset
   405
        return list(self.iter_process_result(cursor, cnx, column_callbacks))
7341
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   406
10086
98bc2ca1a816 [source/native] session -> cnx
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9891
diff changeset
   407
    def iter_process_result(self, cursor, cnx, column_callbacks=None):
7341
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   408
        """return a iterator on tuples of CubicWeb compliant values from data
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   409
        in the given cursor
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   410
        """
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4965
diff changeset
   411
        # use two different implementations to avoid paying the price of
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4965
diff changeset
   412
        # callback lookup for each *cell* in results when there is nothing to
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4965
diff changeset
   413
        # lookup
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4965
diff changeset
   414
        if not column_callbacks:
8139
f9ebb6d1abc3 [server] use lgd.process_cursor to optimize processing of large resultsets
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7907
diff changeset
   415
            return self.dbhelper.dbapi_module.process_cursor(cursor, self._dbencoding,
f9ebb6d1abc3 [server] use lgd.process_cursor to optimize processing of large resultsets
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 7907
diff changeset
   416
                                                             Binary)
10086
98bc2ca1a816 [source/native] session -> cnx
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9891
diff changeset
   417
        assert cnx
98bc2ca1a816 [source/native] session -> cnx
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9891
diff changeset
   418
        return self._cb_process_result(cursor, column_callbacks, cnx)
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4965
diff changeset
   419
10086
98bc2ca1a816 [source/native] session -> cnx
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9891
diff changeset
   420
    def _cb_process_result(self, cursor, column_callbacks, cnx):
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4965
diff changeset
   421
        # begin bind to locals for optimization
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4965
diff changeset
   422
        descr = cursor.description
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4965
diff changeset
   423
        encoding = self._dbencoding
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4965
diff changeset
   424
        process_value = self._process_value
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4965
diff changeset
   425
        binary = Binary
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4965
diff changeset
   426
        # /end
7341
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   427
        cursor.arraysize = 100
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   428
        while True:
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   429
            results = cursor.fetchmany()
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   430
            if not results:
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   431
                break
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   432
            for line in results:
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   433
                result = []
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   434
                for col, value in enumerate(line):
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   435
                    if value is None:
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   436
                        result.append(value)
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   437
                        continue
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   438
                    cbstack = column_callbacks.get(col, None)
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   439
                    if cbstack is None:
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   440
                        value = process_value(value, descr[col], encoding, binary)
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   441
                    else:
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   442
                        for cb in cbstack:
10086
98bc2ca1a816 [source/native] session -> cnx
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9891
diff changeset
   443
                            value = cb(self, cnx, value)
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4965
diff changeset
   444
                    result.append(value)
7341
c419c2d0d13e add a new method iter_process_result which does the same as proces_result but is a generator (closes #1625374)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7229
diff changeset
   445
                yield result
5013
ad91f93bbb93 [source storage] refactor source sql generation and results handling to allow repository side callbacks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4965
diff changeset
   446
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   447
    def preprocess_entity(self, entity):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   448
        """return a dictionary to use as extra argument to cursor.execute
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   449
        to insert/update an entity into a SQL database
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   450
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   451
        attrs = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   452
        eschema = entity.e_schema
8944
b167f039b6cb [sql] preprocess_entity uses lgdb helper's SQL converters.
Vincent Michel <vincent.michel@logilab.fr>
parents: 8755
diff changeset
   453
        converters = getattr(self.dbhelper, 'TYPE_CONVERTERS', {})
10662
10942ed172de [py3k] dict.iteritems → dict.items
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10651
diff changeset
   454
        for attr, value in entity.cw_edited.items():
6284
c35b2ebeb3c9 [repo/sql] don't do anything when value is None. This will avoid None to be turned into False in the case of a boolean for instance.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5768
diff changeset
   455
            if value is not None and eschema.subjrels[attr].final:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   456
                atype = str(entity.e_schema.destination(attr))
8944
b167f039b6cb [sql] preprocess_entity uses lgdb helper's SQL converters.
Vincent Michel <vincent.michel@logilab.fr>
parents: 8755
diff changeset
   457
                if atype in converters:
b167f039b6cb [sql] preprocess_entity uses lgdb helper's SQL converters.
Vincent Michel <vincent.michel@logilab.fr>
parents: 8755
diff changeset
   458
                    # It is easier to modify preprocess_entity rather
b167f039b6cb [sql] preprocess_entity uses lgdb helper's SQL converters.
Vincent Michel <vincent.michel@logilab.fr>
parents: 8755
diff changeset
   459
                    # than add_entity (native) as this behavior
b167f039b6cb [sql] preprocess_entity uses lgdb helper's SQL converters.
Vincent Michel <vincent.michel@logilab.fr>
parents: 8755
diff changeset
   460
                    # may also be used for update.
b167f039b6cb [sql] preprocess_entity uses lgdb helper's SQL converters.
Vincent Michel <vincent.michel@logilab.fr>
parents: 8755
diff changeset
   461
                    value = converters[atype](value)
b167f039b6cb [sql] preprocess_entity uses lgdb helper's SQL converters.
Vincent Michel <vincent.michel@logilab.fr>
parents: 8755
diff changeset
   462
                elif atype == 'Password': # XXX could be done using a TYPE_CONVERTERS callback
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   463
                    # if value is a Binary instance, this mean we got it
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   464
                    # from a query result and so it is already encrypted
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   465
                    if isinstance(value, Binary):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   466
                        value = value.getvalue()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   467
                    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   468
                        value = crypt_password(value)
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   469
                    value = self._binary(value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   470
                elif isinstance(value, Binary):
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   471
                    value = self._binary(value.getvalue())
1251
af40e615dc89 introduce a 'cw_' prefix on entity table and column names so we don't conflict with sql or DBMS specific keywords
sylvain.thenault@logilab.fr
parents: 0
diff changeset
   472
            attrs[SQL_PREFIX+str(attr)] = value
4965
04543ed0bbdc [source] only consider edited_attributes in source.preprocess_entity()
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 4913
diff changeset
   473
        attrs[SQL_PREFIX+'eid'] = entity.eid
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   474
        return attrs
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   475
7083
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6379
diff changeset
   476
    # these are overridden by set_log_methods below
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6379
diff changeset
   477
    # only defining here to prevent pylint from complaining
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6379
diff changeset
   478
    info = warning = error = critical = exception = debug = lambda msg,*a,**kw: None
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   479
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   480
set_log_methods(SQLAdapterMixIn, getLogger('cubicweb.sqladapter'))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   481
9447
0636c4960259 [multi-sources-removal] Drop cw.server.sources.extlite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9364
diff changeset
   482
9466
c3a5f4507f12 [multi-sources-removal] Turn ConnectionsSet into simpler ConnectionWrapper
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9447
diff changeset
   483
# connection initialization functions ##########################################
9447
0636c4960259 [multi-sources-removal] Drop cw.server.sources.extlite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9364
diff changeset
   484
11030
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   485
def _install_sqlite_querier_patch():
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   486
    """This monkey-patch hotfixes a bug sqlite causing some dates to be returned as strings rather than
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   487
    date objects (http://www.sqlite.org/cvstrac/tktview?tn=1327,33)
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   488
    """
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   489
    from cubicweb.server.querier import QuerierHelper
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   490
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   491
    if hasattr(QuerierHelper, '_sqlite_patched'):
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   492
        return  # already monkey patched
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   493
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   494
    def wrap_execute(base_execute):
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   495
        def new_execute(*args, **kwargs):
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   496
            rset = base_execute(*args, **kwargs)
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   497
            if rset.description:
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   498
                found_date = False
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   499
                for row, rowdesc in zip(rset, rset.description):
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   500
                    for cellindex, (value, vtype) in enumerate(zip(row, rowdesc)):
11032
38afb7e23c6c [server] fix SQLite patch to ensure TZDatetime are properly typed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11031
diff changeset
   501
                        if vtype in ('TZDatetime', 'Date', 'Datetime') \
38afb7e23c6c [server] fix SQLite patch to ensure TZDatetime are properly typed
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11031
diff changeset
   502
                           and isinstance(value, text_type):
11030
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   503
                            found_date = True
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   504
                            value = value.rsplit('.', 1)[0]
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   505
                            try:
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   506
                                row[cellindex] = strptime(value, '%Y-%m-%d %H:%M:%S')
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   507
                            except Exception:
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   508
                                row[cellindex] = strptime(value, '%Y-%m-%d')
11034
75d752e6daf7 [server] improve TZDatetime support
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11032
diff changeset
   509
                            if vtype == 'TZDatetime':
75d752e6daf7 [server] improve TZDatetime support
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11032
diff changeset
   510
                                row[cellindex] = row[cellindex].replace(tzinfo=utc)
11030
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   511
                        if vtype == 'Time' and isinstance(value, text_type):
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   512
                            found_date = True
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   513
                            try:
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   514
                                row[cellindex] = strptime(value, '%H:%M:%S')
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   515
                            except Exception:
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   516
                                # DateTime used as Time?
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   517
                                row[cellindex] = strptime(value, '%Y-%m-%d %H:%M:%S')
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   518
                        if vtype == 'Interval' and isinstance(value, int):
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   519
                            found_date = True
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   520
                            # XXX value is in number of seconds?
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   521
                            row[cellindex] = timedelta(0, value, 0)
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   522
                    if not found_date:
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   523
                        break
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   524
            return rset
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   525
        return new_execute
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   526
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   527
    QuerierHelper.execute = wrap_execute(QuerierHelper.execute)
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   528
    QuerierHelper._sqlite_patched = True
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   529
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   530
11031
780939fc06da [server] fix name and add docstring to connection initialization functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11030
diff changeset
   531
def _init_sqlite_connection(cnx):
780939fc06da [server] fix name and add docstring to connection initialization functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11030
diff changeset
   532
    """Internal function that will be called to init a sqlite connection"""
11030
c1fdd22232d1 [server] always monkeypatch QuerierHelper to handle dates and times on SQLite
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10824
diff changeset
   533
    _install_sqlite_querier_patch()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   534
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   535
    class group_concat(object):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   536
        def __init__(self):
9364
73bd5012336f Make the GROUP_CONCAT aggregate function not repeat values (closes #3223975)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9340
diff changeset
   537
            self.values = set()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   538
        def step(self, value):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   539
            if value is not None:
9364
73bd5012336f Make the GROUP_CONCAT aggregate function not repeat values (closes #3223975)
Julien Cristau <julien.cristau@logilab.fr>
parents: 9340
diff changeset
   540
                self.values.add(value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   541
        def finalize(self):
10760
c34bbdb18745 [server] unicode → six.text_type
Julien Cristau <julien.cristau@logilab.fr>
parents: 10662
diff changeset
   542
            return ', '.join(text_type(v) for v in self.values)
9334
ea12401c0a68 [sqlutils] avoid a crash with sqlite when using group_concat (closes #3331906)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9267
diff changeset
   543
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   544
    cnx.create_aggregate("GROUP_CONCAT", 1, group_concat)
1619
e4845b54a704 force proper date/datetime according to type (necessary for sqlite at least)
sylvain.thenault@logilab.fr
parents: 1408
diff changeset
   545
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   546
    def _limit_size(text, maxsize, format='text/plain'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   547
        if len(text) < maxsize:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   548
            return text
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   549
        if format in ('text/html', 'text/xhtml', 'text/xml'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   550
            text = remove_html_tags(text)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   551
        if len(text) > maxsize:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   552
            text = text[:maxsize] + '...'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   553
        return text
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   554
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   555
    def limit_size3(text, format, maxsize):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   556
        return _limit_size(text, maxsize, format)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   557
    cnx.create_function("LIMIT_SIZE", 3, limit_size3)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   558
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   559
    def limit_size2(text, maxsize):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   560
        return _limit_size(text, maxsize)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   561
    cnx.create_function("TEXT_LIMIT_SIZE", 2, limit_size2)
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   562
7904
f41bb38dda7c [rql, sql] support for weekday function introduced in lgdp 1.7 (closes #1979717)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7341
diff changeset
   563
    from logilab.common.date import strptime
f41bb38dda7c [rql, sql] support for weekday function introduced in lgdp 1.7 (closes #1979717)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7341
diff changeset
   564
    def weekday(ustr):
f41bb38dda7c [rql, sql] support for weekday function introduced in lgdp 1.7 (closes #1979717)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7341
diff changeset
   565
        try:
f41bb38dda7c [rql, sql] support for weekday function introduced in lgdp 1.7 (closes #1979717)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7341
diff changeset
   566
            dt = strptime(ustr, '%Y-%m-%d %H:%M:%S')
f41bb38dda7c [rql, sql] support for weekday function introduced in lgdp 1.7 (closes #1979717)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7341
diff changeset
   567
        except:
f41bb38dda7c [rql, sql] support for weekday function introduced in lgdp 1.7 (closes #1979717)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7341
diff changeset
   568
            dt =  strptime(ustr, '%Y-%m-%d')
f41bb38dda7c [rql, sql] support for weekday function introduced in lgdp 1.7 (closes #1979717)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7341
diff changeset
   569
        # expect sunday to be 1, saturday 7 while weekday method return 0 for
f41bb38dda7c [rql, sql] support for weekday function introduced in lgdp 1.7 (closes #1979717)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7341
diff changeset
   570
        # monday
f41bb38dda7c [rql, sql] support for weekday function introduced in lgdp 1.7 (closes #1979717)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7341
diff changeset
   571
        return (dt.weekday() + 1) % 7
f41bb38dda7c [rql, sql] support for weekday function introduced in lgdp 1.7 (closes #1979717)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7341
diff changeset
   572
    cnx.create_function("WEEKDAY", 1, weekday)
f41bb38dda7c [rql, sql] support for weekday function introduced in lgdp 1.7 (closes #1979717)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7341
diff changeset
   573
10205
737271ffe407 Ask sqlite to check foreign keys
Julien Cristau <julien.cristau@logilab.fr>
parents: 10200
diff changeset
   574
    cnx.cursor().execute("pragma foreign_keys = on")
737271ffe407 Ask sqlite to check foreign keys
Julien Cristau <julien.cristau@logilab.fr>
parents: 10200
diff changeset
   575
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   576
    import yams.constraints
4831
c5aec27c1bf7 [repo] use logilab.db instead of lgc.adbh/lgc.db/lgc.sqlgen/indexer, test new date extranction functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
   577
    yams.constraints.patch_sqlite_decimal()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   578
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   579
sqlite_hooks = SQL_CONNECT_HOOKS.setdefault('sqlite', [])
11031
780939fc06da [server] fix name and add docstring to connection initialization functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11030
diff changeset
   580
sqlite_hooks.append(_init_sqlite_connection)
7166
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7083
diff changeset
   581
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7083
diff changeset
   582
11031
780939fc06da [server] fix name and add docstring to connection initialization functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11030
diff changeset
   583
def _init_postgres_connection(cnx):
780939fc06da [server] fix name and add docstring to connection initialization functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11030
diff changeset
   584
    """Internal function that will be called to init a postgresql connection"""
7166
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7083
diff changeset
   585
    cnx.cursor().execute('SET TIME ZONE UTC')
7229
a60522259c2c [tz postgres support] we've to commit the connection once time-zone is set, else we may loose the setting
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7221
diff changeset
   586
    # commit is needed, else setting are lost if the connection is first
9267
24d9b86dfa54 spelling: rollbacked -> rolled back
Rémi Cardona <remi.cardona@logilab.fr>
parents: 8944
diff changeset
   587
    # rolled back
7229
a60522259c2c [tz postgres support] we've to commit the connection once time-zone is set, else we may loose the setting
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7221
diff changeset
   588
    cnx.commit()
7166
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7083
diff changeset
   589
dde161937d3e [time zone] support for TZDatetime and TZTime data type
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7083
diff changeset
   590
postgres_hooks = SQL_CONNECT_HOOKS.setdefault('postgres', [])
11031
780939fc06da [server] fix name and add docstring to connection initialization functions
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11030
diff changeset
   591
postgres_hooks.append(_init_postgres_connection)