cubicweb/server/utils.py
author Denis Laxalde <denis.laxalde@logilab.fr>
Mon, 06 Mar 2017 13:23:33 +0100
changeset 12012 f7ff5217a02f
parent 12011 d2888fee6031
child 12013 7b975655d0ae
permissions -rw-r--r--
[server] replace repository's tasks manager by a scheduler Repository() does not accept anymore a 'tasks_manager' argument but rather a 'scheduler' argument which is expected to be an instance of sched.scheduler class. The drop the _tasks_manager attribute of the repository and adjust all internal usages of it. In particular, in the 'repo_stats' service we do not export 'looping_tasks' statistics anymore as there's no way to retrieve this anymore from a web instance. Closes #17057223.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
11091
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
     1
# copyright 2003-2016 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5606
diff changeset
    18
"""Some utilities for the CubicWeb server."""
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10095
diff changeset
    19
from __future__ import print_function
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7573
diff changeset
    20
11767
432f87a63057 flake8 and all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11129
diff changeset
    21
12011
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
    22
from functools import wraps
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
    23
import sched
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
import sys
6765
b922e3a817e9 fix ticket #1382716 (problem was actually more subtle than I originally thought)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6427
diff changeset
    25
import logging
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
from threading import Timer, Thread
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
from getpass import getpass
2105
92ea410806fe refactor sources configuration, add source to sources when using a cube defining
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    28
10678
77333ec71fab [server/utils] convert to py3k using six
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    29
from six import PY2, text_type
77333ec71fab [server/utils] convert to py3k using six
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    30
from six.moves import input
77333ec71fab [server/utils] convert to py3k using six
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    31
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    32
from passlib.utils import handlers as uh, to_hash_str
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    33
from passlib.context import CryptContext
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    34
11091
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    35
from logilab.common.deprecation import deprecated
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    36
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    37
from cubicweb.md5crypt import crypt as md5crypt
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    38
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    39
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    40
class CustomMD5Crypt(uh.HasSalt, uh.GenericHandler):
8398
a9fe30c953be [server/utils] passlib 1.6 is now less tolerant wrt handler names (closes #2349330)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8317
diff changeset
    41
    name = 'cubicwebmd5crypt'
a9fe30c953be [server/utils] passlib 1.6 is now less tolerant wrt handler names (closes #2349330)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8317
diff changeset
    42
    setting_kwds = ('salt',)
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    43
    min_salt_size = 0
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    44
    max_salt_size = 8
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    45
    salt_chars = uh.H64_CHARS
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    46
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    47
    @classmethod
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    48
    def from_string(cls, hash):
8414
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    49
        salt, chk = uh.parse_mc2(hash, u'')
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    50
        if chk is None:
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    51
            raise ValueError('missing checksum')
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    52
        return cls(salt=salt, checksum=chk)
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    53
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    54
    def to_string(self):
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    55
        return to_hash_str(u'%s$%s' % (self.salt, self.checksum or u''))
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    56
8414
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    57
    # passlib 1.5 wants calc_checksum, 1.6 wants _calc_checksum
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    58
    def calc_checksum(self, secret):
8414
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    59
        return md5crypt(secret, self.salt.encode('ascii')).decode('utf-8')
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    60
    _calc_checksum = calc_checksum
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    61
8546
3d2038d6f20d [sources/native] automatically update passwords using deprecated hashes on login
Julien Cristau <julien.cristau@logilab.fr>
parents: 8446
diff changeset
    62
_CRYPTO_CTX = CryptContext(['sha512_crypt', CustomMD5Crypt, 'des_crypt', 'ldap_salted_sha1'],
3d2038d6f20d [sources/native] automatically update passwords using deprecated hashes on login
Julien Cristau <julien.cristau@logilab.fr>
parents: 8446
diff changeset
    63
                           deprecated=['cubicwebmd5crypt', 'des_crypt'])
3d2038d6f20d [sources/native] automatically update passwords using deprecated hashes on login
Julien Cristau <julien.cristau@logilab.fr>
parents: 8446
diff changeset
    64
verify_and_update = _CRYPTO_CTX.verify_and_update
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    65
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    66
def crypt_password(passwd, salt=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    67
    """return the encrypted password using the given salt or a generated one
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    68
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    69
    if salt is None:
11880
bc9d901cb9e6 [server] Use CryptContext's hash method instead of deprecated encrypt method
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11767
diff changeset
    70
        return _CRYPTO_CTX.hash(passwd).encode('ascii')
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    71
    # empty hash, accept any password for backwards compat
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    72
    if salt == '':
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    73
        return salt
8550
63260486de89 [server/utils] catch ValueError from password verification
Julien Cristau <julien.cristau@logilab.fr>
parents: 8546
diff changeset
    74
    try:
63260486de89 [server/utils] catch ValueError from password verification
Julien Cristau <julien.cristau@logilab.fr>
parents: 8546
diff changeset
    75
        if _CRYPTO_CTX.verify(passwd, salt):
63260486de89 [server/utils] catch ValueError from password verification
Julien Cristau <julien.cristau@logilab.fr>
parents: 8546
diff changeset
    76
            return salt
63260486de89 [server/utils] catch ValueError from password verification
Julien Cristau <julien.cristau@logilab.fr>
parents: 8546
diff changeset
    77
    except ValueError: # e.g. couldn't identify hash
63260486de89 [server/utils] catch ValueError from password verification
Julien Cristau <julien.cristau@logilab.fr>
parents: 8546
diff changeset
    78
        pass
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    79
    # wrong password
10681
4383f5a30504 [server/utils] crypt_password should always return bytes
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10678
diff changeset
    80
    return b''
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    81
11091
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    82
@deprecated('[3.22] no more necessary, directly get eschema.eid')
9610
29450466273a [server] eschema_eid needs a connection, not a session
Julien Cristau <julien.cristau@logilab.fr>
parents: 9468
diff changeset
    83
def eschema_eid(cnx, eschema):
11091
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    84
    """get eid of the CWEType entity for the given yams type.
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    85
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    86
    This used to be necessary because when the schema has been loaded from the
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    87
    file-system, not from the database, (e.g. during tests), eschema.eid was
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    88
    not set.
5066
bf5cbc351e99 [repo] move eschema_eid function from hooks.metadata to server.utils
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4714
diff changeset
    89
    """
bf5cbc351e99 [repo] move eschema_eid function from hooks.metadata to server.utils
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4714
diff changeset
    90
    return eschema.eid
bf5cbc351e99 [repo] move eschema_eid function from hooks.metadata to server.utils
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4714
diff changeset
    91
bf5cbc351e99 [repo] move eschema_eid function from hooks.metadata to server.utils
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4714
diff changeset
    92
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
DEFAULT_MSG = 'we need a manager connection on the repository \
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    94
(the server doesn\'t have to run, even should better not)'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    95
1910
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
    96
def manager_userpasswd(user=None, msg=DEFAULT_MSG, confirm=False,
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
    97
                       passwdmsg='password'):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    98
    if not user:
3701
104b7c326172 check we've some message to display
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3585
diff changeset
    99
        if msg:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10095
diff changeset
   100
            print(msg)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   101
        while not user:
10678
77333ec71fab [server/utils] convert to py3k using six
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   102
            user = input('login: ')
77333ec71fab [server/utils] convert to py3k using six
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   103
        if PY2:
77333ec71fab [server/utils] convert to py3k using six
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   104
            user = unicode(user, sys.stdin.encoding)
1910
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   105
    passwd = getpass('%s: ' % passwdmsg)
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   106
    if confirm:
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   107
        while True:
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   108
            passwd2 = getpass('confirm password: ')
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   109
            if passwd == passwd2:
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   110
                break
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10095
diff changeset
   111
            print('password doesn\'t match')
1910
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   112
            passwd = getpass('password: ')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   113
    # XXX decode password using stdin encoding then encode it using appl'encoding
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   114
    return user, passwd
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   115
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   116
12011
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   117
if PY2:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   118
    import time  # noqa
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   119
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   120
    class scheduler(sched.scheduler):
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   121
        """Python2 version of sched.scheduler that matches Python3 API."""
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   122
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   123
        def __init__(self, **kwargs):
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   124
            kwargs.setdefault('timefunc', time.time)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   125
            kwargs.setdefault('delayfunc', time.sleep)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   126
            # sched.scheduler is an old-style class.
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   127
            sched.scheduler.__init__(self, **kwargs)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   128
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   129
else:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   130
    scheduler = sched.scheduler
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   131
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   132
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   133
def schedule_periodic_task(scheduler, interval, func, *args):
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   134
    """Enter a task with `func(*args)` as a periodic event in `scheduler`
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   135
    executing at `interval` seconds. Once executed, the task would re-schedule
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   136
    itself unless a BaseException got raised.
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   137
    """
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   138
    @wraps(func)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   139
    def task(*args):
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   140
        restart = True
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   141
        try:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   142
            func(*args)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   143
        except Exception:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   144
            logger = logging.getLogger('cubicweb.scheduler')
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   145
            logger.exception('Unhandled exception in periodic task "%s"',
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   146
                             func.__name__)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   147
        except BaseException as exc:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   148
            logger = logging.getLogger('cubicweb.scheduler')
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   149
            logger.error('periodic task "%s" not re-scheduled due to %r',
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   150
                         func.__name__, exc)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   151
            restart = False
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   152
        finally:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   153
            if restart:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   154
                scheduler.enter(interval, 1, task, argument=args)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   155
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   156
    return scheduler.enter(interval, 1, task, argument=args)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   157
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   158
9468
39b7a91a3f4c [repo] pylint cleanup, mainly of imports, with a bit of style
Julien Cristau <julien.cristau@logilab.fr>
parents: 9467
diff changeset
   159
_MARKER = object()
6381
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   160
def func_name(func):
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   161
    name = getattr(func, '__name__', _MARKER)
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   162
    if name is _MARKER:
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   163
        name = getattr(func, 'func_name', _MARKER)
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   164
    if name is _MARKER:
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   165
        name = repr(func)
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   166
    return name
2105
92ea410806fe refactor sources configuration, add source to sources when using a cube defining
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   167
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   168
class LoopTask(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   169
    """threaded task restarting itself once executed"""
8320
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   170
    def __init__(self, tasks_manager, interval, func, args):
8911
2dd134a86996 [task] allow interval=0 for looping tasks
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8887
diff changeset
   171
        if interval < 0:
2dd134a86996 [task] allow interval=0 for looping tasks
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8887
diff changeset
   172
            raise ValueError('Loop task interval must be >= 0 '
5602
277b15d6d3ed forbid looping tasks with an interval of 0
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5601
diff changeset
   173
                             '(current value: %f for %s)' % \
6381
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   174
                             (interval, func_name(func)))
8320
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   175
        self._tasks_manager = tasks_manager
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   176
        self.interval = interval
2708
60d728bdcba5 allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2105
diff changeset
   177
        def auto_restart_func(self=self, func=func, args=args):
7573
c8f8762c986d [repo, looping task] raise a custom exception when repository is shuting down, avoid looping task to be restarted in such case. Closes #1021276
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6765
diff changeset
   178
            restart = True
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   179
            try:
2708
60d728bdcba5 allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2105
diff changeset
   180
                func(*args)
7573
c8f8762c986d [repo, looping task] raise a custom exception when repository is shuting down, avoid looping task to be restarted in such case. Closes #1021276
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6765
diff changeset
   181
            except Exception:
6765
b922e3a817e9 fix ticket #1382716 (problem was actually more subtle than I originally thought)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6427
diff changeset
   182
                logger = logging.getLogger('cubicweb.repository')
b922e3a817e9 fix ticket #1382716 (problem was actually more subtle than I originally thought)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6427
diff changeset
   183
                logger.exception('Unhandled exception in LoopTask %s', self.name)
b922e3a817e9 fix ticket #1382716 (problem was actually more subtle than I originally thought)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6427
diff changeset
   184
                raise
7573
c8f8762c986d [repo, looping task] raise a custom exception when repository is shuting down, avoid looping task to be restarted in such case. Closes #1021276
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6765
diff changeset
   185
            except BaseException:
c8f8762c986d [repo, looping task] raise a custom exception when repository is shuting down, avoid looping task to be restarted in such case. Closes #1021276
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6765
diff changeset
   186
                restart = False
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   187
            finally:
8320
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   188
                if restart and tasks_manager.running:
7573
c8f8762c986d [repo, looping task] raise a custom exception when repository is shuting down, avoid looping task to be restarted in such case. Closes #1021276
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6765
diff changeset
   189
                    self.start()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   190
        self.func = auto_restart_func
6381
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   191
        self.name = func_name(func)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1138
diff changeset
   192
4714
fccda6dd91bf merge debug and info views
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
   193
    def __str__(self):
fccda6dd91bf merge debug and info views
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
   194
        return '%s (%s seconds)' % (self.name, self.interval)
fccda6dd91bf merge debug and info views
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4212
diff changeset
   195
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   196
    def start(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   197
        self._t = Timer(self.interval, self.func)
5601
92cf309672ca /siteinfo page: display information about the names of the running threads
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5424
diff changeset
   198
        self._t.setName('%s-%s[%d]' % (self._t.getName(), self.name, self.interval))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   199
        self._t.start()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   200
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   201
    def cancel(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   202
        self._t.cancel()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   203
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   204
    def join(self):
5581
0aae5216f99e [repo] ensure thread is alive before calling .join. Closes #963580
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
   205
        if self._t.isAlive():
0aae5216f99e [repo] ensure thread is alive before calling .join. Closes #963580
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5424
diff changeset
   206
            self._t.join()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   207
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   208
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   209
class RepoThread(Thread):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   210
    """subclass of thread so it auto remove itself from a given list once
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   211
    executed
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   212
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   213
    def __init__(self, target, running_threads):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   214
        def auto_remove_func(self=self, func=target):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   215
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   216
                func()
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7573
diff changeset
   217
            except Exception:
6765
b922e3a817e9 fix ticket #1382716 (problem was actually more subtle than I originally thought)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6427
diff changeset
   218
                logger = logging.getLogger('cubicweb.repository')
b922e3a817e9 fix ticket #1382716 (problem was actually more subtle than I originally thought)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6427
diff changeset
   219
                logger.exception('Unhandled exception in RepoThread %s', self._name)
b922e3a817e9 fix ticket #1382716 (problem was actually more subtle than I originally thought)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6427
diff changeset
   220
                raise
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   221
            finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   222
                self.running_threads.remove(self)
1138
22f634977c95 make pylint happy, fix some bugs on the way
sylvain.thenault@logilab.fr
parents: 1134
diff changeset
   223
        Thread.__init__(self, target=auto_remove_func)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   224
        self.running_threads = running_threads
6381
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   225
        self._name = func_name(target)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1138
diff changeset
   226
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   227
    def start(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   228
        self.running_threads.append(self)
3585
cd437d24aa65 use daemon thread
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2708
diff changeset
   229
        self.daemon = True
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   230
        Thread.start(self)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   231
5376
2c3f14bc2590 [python2.6] don't add a name property on Thread
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5066
diff changeset
   232
    def getName(self):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   233
        return '%s(%s)' % (self._name, Thread.getName(self))
8320
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   234
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   235
class TasksManager(object):
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   236
    """Object dedicated manage background task"""
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   237
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   238
    def __init__(self):
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   239
        self.running = False
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   240
        self._tasks = []
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   241
        self._looping_tasks = []
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   242
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   243
    def add_looping_task(self, interval, func, *args):
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   244
        """register a function to be called every `interval` seconds.
8912
da2007002dca [task] allow negative intervals for add_looping_task (closes #2818280)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8911
diff changeset
   245
da2007002dca [task] allow negative intervals for add_looping_task (closes #2818280)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8911
diff changeset
   246
        If interval is negative, no looping task is registered.
8320
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   247
        """
8912
da2007002dca [task] allow negative intervals for add_looping_task (closes #2818280)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8911
diff changeset
   248
        if interval < 0:
da2007002dca [task] allow negative intervals for add_looping_task (closes #2818280)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8911
diff changeset
   249
            self.debug('looping task %s ignored due to interval %f < 0',
da2007002dca [task] allow negative intervals for add_looping_task (closes #2818280)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8911
diff changeset
   250
                       func_name(func), interval)
da2007002dca [task] allow negative intervals for add_looping_task (closes #2818280)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8911
diff changeset
   251
            return
8446
cae198371548 [server/utils] add_looping_task can be used any time (closes #2408814)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8425
diff changeset
   252
        task = LoopTask(self, interval, func, args)
8320
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   253
        if self.running:
8446
cae198371548 [server/utils] add_looping_task can be used any time (closes #2408814)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8425
diff changeset
   254
            self._start_task(task)
cae198371548 [server/utils] add_looping_task can be used any time (closes #2408814)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8425
diff changeset
   255
        else:
cae198371548 [server/utils] add_looping_task can be used any time (closes #2408814)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8425
diff changeset
   256
            self._tasks.append(task)
cae198371548 [server/utils] add_looping_task can be used any time (closes #2408814)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8425
diff changeset
   257
cae198371548 [server/utils] add_looping_task can be used any time (closes #2408814)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8425
diff changeset
   258
    def _start_task(self, task):
cae198371548 [server/utils] add_looping_task can be used any time (closes #2408814)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8425
diff changeset
   259
        self._looping_tasks.append(task)
cae198371548 [server/utils] add_looping_task can be used any time (closes #2408814)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8425
diff changeset
   260
        self.info('starting task %s with interval %.2fs', task.name,
cae198371548 [server/utils] add_looping_task can be used any time (closes #2408814)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8425
diff changeset
   261
                  task.interval)
cae198371548 [server/utils] add_looping_task can be used any time (closes #2408814)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8425
diff changeset
   262
        task.start()
8320
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   263
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   264
    def start(self):
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   265
        """Start running looping task"""
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   266
        assert self.running == False # bw compat purpose maintly
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   267
        while self._tasks:
8446
cae198371548 [server/utils] add_looping_task can be used any time (closes #2408814)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8425
diff changeset
   268
            task = self._tasks.pop()
cae198371548 [server/utils] add_looping_task can be used any time (closes #2408814)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8425
diff changeset
   269
            self._start_task(task)
8320
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   270
        self.running = True
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   271
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   272
    def stop(self):
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   273
        """Stop all running task.
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   274
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   275
        returns when all task have been cancel and none are running anymore"""
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   276
        if self.running:
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   277
            while self._looping_tasks:
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   278
                looptask = self._looping_tasks.pop()
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   279
                self.info('canceling task %s...', looptask.name)
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   280
                looptask.cancel()
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   281
                looptask.join()
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   282
                self.info('task %s finished', looptask.name)
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   283
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   284
from logging import getLogger
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   285
from cubicweb import set_log_methods
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 7823
diff changeset
   286
set_log_methods(TasksManager, getLogger('cubicweb.repository'))