cubicweb/server/utils.py
author Denis Laxalde <denis.laxalde@logilab.fr>
Fri, 05 Apr 2019 17:58:19 +0200
changeset 12567 26744ad37953
parent 12508 a8c1ea390400
child 12574 6ccf8fda063f
permissions -rw-r--r--
Drop python2 support This mostly consists in removing the dependency on "six" and updating the code to use only Python3 idioms. Notice that we previously used TemporaryDirectory from cubicweb.devtools.testlib for compatibility with Python2. We now directly import it from tempfile.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
11091
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
     1
# copyright 2003-2016 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5606
diff changeset
    18
"""Some utilities for the CubicWeb server."""
11767
432f87a63057 flake8 and all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11129
diff changeset
    19
12011
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
    20
from functools import wraps
6765
b922e3a817e9 fix ticket #1382716 (problem was actually more subtle than I originally thought)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6427
diff changeset
    21
import logging
12013
7b975655d0ae [server] Drop utils's LoopTask and TasksManager classes not used anymore
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12011
diff changeset
    22
from threading import Thread
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
from getpass import getpass
2105
92ea410806fe refactor sources configuration, add source to sources when using a cube defining
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    24
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    25
from passlib.utils import handlers as uh, to_hash_str
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    26
from passlib.context import CryptContext
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    27
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    28
from cubicweb.md5crypt import crypt as md5crypt
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    29
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    30
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    31
class CustomMD5Crypt(uh.HasSalt, uh.GenericHandler):
8398
a9fe30c953be [server/utils] passlib 1.6 is now less tolerant wrt handler names (closes #2349330)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8317
diff changeset
    32
    name = 'cubicwebmd5crypt'
a9fe30c953be [server/utils] passlib 1.6 is now less tolerant wrt handler names (closes #2349330)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8317
diff changeset
    33
    setting_kwds = ('salt',)
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    34
    min_salt_size = 0
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    35
    max_salt_size = 8
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    36
    salt_chars = uh.H64_CHARS
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    37
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    38
    @classmethod
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    39
    def from_string(cls, hash):
8414
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    40
        salt, chk = uh.parse_mc2(hash, u'')
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    41
        if chk is None:
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    42
            raise ValueError('missing checksum')
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    43
        return cls(salt=salt, checksum=chk)
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    44
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    45
    def to_string(self):
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    46
        return to_hash_str(u'%s$%s' % (self.salt, self.checksum or u''))
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    47
12193
e4ee04d442be [server/utils] update passlib API to 1.7 (closes #17054805)
David Douard <david.douard@logilab.fr>
parents: 12014
diff changeset
    48
    def _calc_checksum(self, secret):
8414
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    49
        return md5crypt(secret, self.salt.encode('ascii')).decode('utf-8')
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    50
12014
116005b5fce9 [server] Make cubicweb/server/utils.py flake8-clean
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12013
diff changeset
    51
8546
3d2038d6f20d [sources/native] automatically update passwords using deprecated hashes on login
Julien Cristau <julien.cristau@logilab.fr>
parents: 8446
diff changeset
    52
_CRYPTO_CTX = CryptContext(['sha512_crypt', CustomMD5Crypt, 'des_crypt', 'ldap_salted_sha1'],
3d2038d6f20d [sources/native] automatically update passwords using deprecated hashes on login
Julien Cristau <julien.cristau@logilab.fr>
parents: 8446
diff changeset
    53
                           deprecated=['cubicwebmd5crypt', 'des_crypt'])
12193
e4ee04d442be [server/utils] update passlib API to 1.7 (closes #17054805)
David Douard <david.douard@logilab.fr>
parents: 12014
diff changeset
    54
# for bw compat with passlib < 1.7
e4ee04d442be [server/utils] update passlib API to 1.7 (closes #17054805)
David Douard <david.douard@logilab.fr>
parents: 12014
diff changeset
    55
if not hasattr(_CRYPTO_CTX, 'hash'):
e4ee04d442be [server/utils] update passlib API to 1.7 (closes #17054805)
David Douard <david.douard@logilab.fr>
parents: 12014
diff changeset
    56
    _CRYPTO_CTX.hash = _CRYPTO_CTX.encrypt
8546
3d2038d6f20d [sources/native] automatically update passwords using deprecated hashes on login
Julien Cristau <julien.cristau@logilab.fr>
parents: 8446
diff changeset
    57
verify_and_update = _CRYPTO_CTX.verify_and_update
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    58
12014
116005b5fce9 [server] Make cubicweb/server/utils.py flake8-clean
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12013
diff changeset
    59
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    60
def crypt_password(passwd, salt=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    61
    """return the encrypted password using the given salt or a generated one
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    62
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    63
    if salt is None:
11880
bc9d901cb9e6 [server] Use CryptContext's hash method instead of deprecated encrypt method
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11767
diff changeset
    64
        return _CRYPTO_CTX.hash(passwd).encode('ascii')
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    65
    # empty hash, accept any password for backwards compat
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    66
    if salt == '':
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    67
        return salt
8550
63260486de89 [server/utils] catch ValueError from password verification
Julien Cristau <julien.cristau@logilab.fr>
parents: 8546
diff changeset
    68
    try:
63260486de89 [server/utils] catch ValueError from password verification
Julien Cristau <julien.cristau@logilab.fr>
parents: 8546
diff changeset
    69
        if _CRYPTO_CTX.verify(passwd, salt):
63260486de89 [server/utils] catch ValueError from password verification
Julien Cristau <julien.cristau@logilab.fr>
parents: 8546
diff changeset
    70
            return salt
12014
116005b5fce9 [server] Make cubicweb/server/utils.py flake8-clean
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12013
diff changeset
    71
    except ValueError:  # e.g. couldn't identify hash
8550
63260486de89 [server/utils] catch ValueError from password verification
Julien Cristau <julien.cristau@logilab.fr>
parents: 8546
diff changeset
    72
        pass
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    73
    # wrong password
10681
4383f5a30504 [server/utils] crypt_password should always return bytes
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10678
diff changeset
    74
    return b''
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    75
12014
116005b5fce9 [server] Make cubicweb/server/utils.py flake8-clean
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12013
diff changeset
    76
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    77
DEFAULT_MSG = 'we need a manager connection on the repository \
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    78
(the server doesn\'t have to run, even should better not)'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    79
12014
116005b5fce9 [server] Make cubicweb/server/utils.py flake8-clean
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12013
diff changeset
    80
1910
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
    81
def manager_userpasswd(user=None, msg=DEFAULT_MSG, confirm=False,
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
    82
                       passwdmsg='password'):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    83
    if not user:
3701
104b7c326172 check we've some message to display
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3585
diff changeset
    84
        if msg:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10095
diff changeset
    85
            print(msg)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
        while not user:
10678
77333ec71fab [server/utils] convert to py3k using six
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    87
            user = input('login: ')
1910
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
    88
    passwd = getpass('%s: ' % passwdmsg)
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
    89
    if confirm:
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
    90
        while True:
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
    91
            passwd2 = getpass('confirm password: ')
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
    92
            if passwd == passwd2:
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
    93
                break
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10095
diff changeset
    94
            print('password doesn\'t match')
1910
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
    95
            passwd = getpass('password: ')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
    # XXX decode password using stdin encoding then encode it using appl'encoding
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    97
    return user, passwd
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    98
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    99
12011
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   100
def schedule_periodic_task(scheduler, interval, func, *args):
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   101
    """Enter a task with `func(*args)` as a periodic event in `scheduler`
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   102
    executing at `interval` seconds. Once executed, the task would re-schedule
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   103
    itself unless a BaseException got raised.
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   104
    """
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   105
    @wraps(func)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   106
    def task(*args):
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   107
        restart = True
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   108
        try:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   109
            func(*args)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   110
        except Exception:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   111
            logger = logging.getLogger('cubicweb.scheduler')
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   112
            logger.exception('Unhandled exception in periodic task "%s"',
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   113
                             func.__name__)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   114
        except BaseException as exc:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   115
            logger = logging.getLogger('cubicweb.scheduler')
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   116
            logger.error('periodic task "%s" not re-scheduled due to %r',
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   117
                         func.__name__, exc)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   118
            restart = False
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   119
        finally:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   120
            if restart:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   121
                scheduler.enter(interval, 1, task, argument=args)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   122
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   123
    return scheduler.enter(interval, 1, task, argument=args)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   124
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   125
9468
39b7a91a3f4c [repo] pylint cleanup, mainly of imports, with a bit of style
Julien Cristau <julien.cristau@logilab.fr>
parents: 9467
diff changeset
   126
_MARKER = object()
12014
116005b5fce9 [server] Make cubicweb/server/utils.py flake8-clean
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12013
diff changeset
   127
116005b5fce9 [server] Make cubicweb/server/utils.py flake8-clean
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12013
diff changeset
   128
6381
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   129
def func_name(func):
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   130
    name = getattr(func, '__name__', _MARKER)
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   131
    if name is _MARKER:
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   132
        name = getattr(func, 'func_name', _MARKER)
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   133
    if name is _MARKER:
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   134
        name = repr(func)
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   135
    return name
2105
92ea410806fe refactor sources configuration, add source to sources when using a cube defining
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   136
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   138
class RepoThread(Thread):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   139
    """subclass of thread so it auto remove itself from a given list once
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   140
    executed
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
    def __init__(self, target, running_threads):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   143
        def auto_remove_func(self=self, func=target):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   144
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   145
                func()
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7573
diff changeset
   146
            except Exception:
6765
b922e3a817e9 fix ticket #1382716 (problem was actually more subtle than I originally thought)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6427
diff changeset
   147
                logger = logging.getLogger('cubicweb.repository')
b922e3a817e9 fix ticket #1382716 (problem was actually more subtle than I originally thought)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6427
diff changeset
   148
                logger.exception('Unhandled exception in RepoThread %s', self._name)
b922e3a817e9 fix ticket #1382716 (problem was actually more subtle than I originally thought)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6427
diff changeset
   149
                raise
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   150
            finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   151
                self.running_threads.remove(self)
1138
22f634977c95 make pylint happy, fix some bugs on the way
sylvain.thenault@logilab.fr
parents: 1134
diff changeset
   152
        Thread.__init__(self, target=auto_remove_func)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   153
        self.running_threads = running_threads
6381
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   154
        self._name = func_name(target)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1138
diff changeset
   155
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   156
    def start(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   157
        self.running_threads.append(self)
3585
cd437d24aa65 use daemon thread
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2708
diff changeset
   158
        self.daemon = True
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   159
        Thread.start(self)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
5376
2c3f14bc2590 [python2.6] don't add a name property on Thread
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5066
diff changeset
   161
    def getName(self):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   162
        return '%s(%s)' % (self._name, Thread.getName(self))