server/pool.py
author Sylvain Thénault <sylvain.thenault@logilab.fr>
Thu, 03 Dec 2009 17:17:43 +0100
changeset 3998 94cc7cad3d2d
parent 3720 5376aaadd16b
parent 3986 cc29eddf51ad
child 4252 6c4f109c2b03
permissions -rw-r--r--
backport stable into default
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2765
diff changeset
     1
"""CubicWeb server connections pool : the repository has a limited number of
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2765
diff changeset
     2
connections pools, each of them dealing with a set of connections on each source
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2765
diff changeset
     3
used by the repository. A connections pools (`ConnectionsPool`) is an
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2765
diff changeset
     4
abstraction for a group of connection to each source.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     5
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     6
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     7
:organization: Logilab
1977
606923dff11b big bunch of copyright / docstring update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
     8
:copyright: 2001-2009 LOGILAB S.A. (Paris, FRANCE), license is LGPL v2.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
     9
:contact: http://www.logilab.fr/ -- mailto:contact@logilab.fr
1977
606923dff11b big bunch of copyright / docstring update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
    10
:license: GNU Lesser General Public License, v2.1 - http://www.gnu.org/licenses
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    11
"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    12
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    13
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    14
import sys
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1132
diff changeset
    15
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    16
class ConnectionsPool(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    17
    """handle connections on a set of sources, at some point associated to a
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
    user session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
    def __init__(self, sources):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
        # dictionnary of (source, connection), indexed by sources'uri
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
        self.source_cnxs = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
        for source in sources:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    25
            self.source_cnxs[source.uri] = (source, source.get_connection())
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
        if not 'system' in self.source_cnxs:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
            self.source_cnxs['system'] = self.source_cnxs[sources[0].uri]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    28
        self._cursors = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    29
2765
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
    30
    def __getitem__(self, uri):
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
    31
        """subscription notation provide access to sources'cursors"""
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
    32
        try:
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
    33
            cursor = self._cursors[uri]
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
    34
        except KeyError:
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
    35
            cursor = self.source_cnxs[uri][1].cursor()
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
    36
            if cursor is not None:
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
    37
                # None possible on sources without cursor support such as ldap
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
    38
                self._cursors[uri] = cursor
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
    39
        return cursor
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
    40
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    41
    def commit(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    42
        """commit the current transaction for this user"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    43
        # FIXME: what happends if a commit fail
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    44
        # would need a two phases commit or like, but I don't know how to do
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    45
        # this using the db-api...
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    46
        for source, cnx in self.source_cnxs.values():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    47
            # let exception propagates
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    48
            cnx.commit()
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1132
diff changeset
    49
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
    def rollback(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    51
        """rollback the current transaction for this user"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    52
        for source, cnx in self.source_cnxs.values():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    53
            # catch exceptions, rollback other sources anyway
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    54
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    55
                cnx.rollback()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    56
            except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    57
                source.critical('rollback error', exc_info=sys.exc_info())
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    58
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    59
    def close(self, i_know_what_i_do=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    60
        """close all connections in the pool"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    61
        if i_know_what_i_do is not True: # unexpected closing safety belt
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    62
            raise RuntimeError('pool shouldn\'t be closed')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
        for cu in self._cursors.values():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    64
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    65
                cu.close()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    66
            except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    67
                continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    68
        for _, cnx in self.source_cnxs.values():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    69
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    70
                cnx.close()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    71
            except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    72
                continue
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1132
diff changeset
    73
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    74
    # internals ###############################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    75
2063
fe4278b50388 fix [re]set_pool prototype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    76
    def pool_set(self):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    77
        """pool is being set"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    78
        self.check_connections()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    79
2063
fe4278b50388 fix [re]set_pool prototype
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    80
    def pool_reset(self):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    81
        """pool is being reseted"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    82
        for source, cnx in self.source_cnxs.values():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    83
            source.pool_reset(cnx)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1132
diff changeset
    84
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    85
    def sources(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
        """return the source objects handled by this pool"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
        # implementation details of flying insert requires the system source
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    88
        # first
2765
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
    89
        yield self.source_cnxs['system'][0]
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
        for uri, (source, cursor) in self.source_cnxs.items():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
            if uri == 'system':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
                continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
            yield source
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    94
        #return [source_cnx[0] for source_cnx in self.source_cnxs.values()]
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1132
diff changeset
    95
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
    def source(self, uid):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    97
        """return the source object with the given uri"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    98
        return self.source_cnxs[uid][0]
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1132
diff changeset
    99
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   100
    def connection(self, uid):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   101
        """return the connection on the source object with the given uri"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   102
        return self.source_cnxs[uid][1]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   103
2765
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
   104
    def reconnect(self, source=None):
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
   105
        """reopen a connection for this source or all sources if none specified
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
   106
        """
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
   107
        if source is None:
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
   108
            sources = self.sources()
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
   109
        else:
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
   110
            sources = (source,)
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
   111
        for source in sources:
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
   112
            source.info('trying to reconnect')
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
   113
            self.source_cnxs[source.uri] = (source, source.get_connection())
5e2525d7b1b1 reconnect without argument reconnect all sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2063
diff changeset
   114
            self._cursors.pop(source.uri, None)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   115
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   116
    def check_connections(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   117
        for source, cnx in self.source_cnxs.itervalues():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   118
            newcnx = source.check_connection(cnx)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   119
            if newcnx is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   120
                self.reset_connection(source, newcnx)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   121
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   122
    def reset_connection(self, source, cnx):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   123
        self.source_cnxs[source.uri] = (source, cnx)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   124
        self._cursors.pop(source.uri, None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   125
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   126
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2765
diff changeset
   127
from cubicweb.server.hook import (Operation, LateOperation, SingleOperation,
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2765
diff changeset
   128
                                  SingleLastOperation)
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2765
diff changeset
   129
from logilab.common.deprecation import class_moved, class_renamed
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2765
diff changeset
   130
Operation = class_moved(Operation)
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2765
diff changeset
   131
PreCommitOperation = class_renamed('PreCommitOperation', Operation)
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2765
diff changeset
   132
LateOperation = class_moved(LateOperation)
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2765
diff changeset
   133
SingleOperation = class_moved(SingleOperation)
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2765
diff changeset
   134
SingleLastOperation = class_moved(SingleLastOperation)