cubicweb/server/utils.py
author Denis Laxalde <denis.laxalde@logilab.fr>
Tue, 04 Apr 2017 16:28:50 +0200
branch3.25
changeset 12142 db2fc87348ab
parent 12014 116005b5fce9
child 12193 e4ee04d442be
permissions -rw-r--r--
[server] Make "sources_by_uri" and "sources_by_eid" properties of repository I.e. do not populate these dict as repo initialization (bootstrap step) but always use information from database. This is needed because when multiple instances of the same application run, if one instance adds a CWSource the other ones will not see it. In particular, when using a scheduler instance, new CWSource will be added by the web instance and not seen by the scheduler which is supposed to update them. We thus define properties for sources_by_eid and sources_by_uri instead attributes on repository instance. CWSource entities are thus retrieved from database every time these properties are accessed. We factor out initialization of the "source" instance (subclass of cubicweb.server.source.AbstractSource) in a _sources() method. Note that this method takes care of calling "init" method on the source as well as "set_schema" (previously done in repo.set_schema(), which now only touches system_source). Accordingly the "init_sources_from_database" method is dropped along with "add_source"/"remove_source" methods. In syncsources hook, we thus drop: * SourceAddedOp operation which called repo.add_source() so that the SourceAddedHook only cares about checking source configuration now; * SourceRemovedOp and SourceRenamedOp operations for the same reason; * SourceConfigUpdatedOp as updating the live config of source is meaningless once we rely on them being retrieved from the database; * SourceHostConfigUpdatedHook hook which is now useless without call to SourceConfigUpdatedOp; In 3.10 migration script, remove usage of sources_by_uri repo attribute which, unless I'm missing something, appears useless (at least now). In tests: * unittest_datafeed: remove test_update_url method since we dropped respective hook; * unittest_ldapsource: LDAPFeedUserDeletionTC.test_a_filter_inactivate() currently fails because it still relies on live config being updated, this will be fixed in the next changeset once all "live source" logic will be removed.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
11091
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
     1
# copyright 2003-2016 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5606
diff changeset
    18
"""Some utilities for the CubicWeb server."""
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10095
diff changeset
    19
from __future__ import print_function
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7573
diff changeset
    20
11767
432f87a63057 flake8 and all
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 11129
diff changeset
    21
12011
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
    22
from functools import wraps
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
    23
import sched
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
import sys
6765
b922e3a817e9 fix ticket #1382716 (problem was actually more subtle than I originally thought)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6427
diff changeset
    25
import logging
12013
7b975655d0ae [server] Drop utils's LoopTask and TasksManager classes not used anymore
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12011
diff changeset
    26
from threading import Thread
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
from getpass import getpass
2105
92ea410806fe refactor sources configuration, add source to sources when using a cube defining
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
    28
12014
116005b5fce9 [server] Make cubicweb/server/utils.py flake8-clean
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12013
diff changeset
    29
from six import PY2
10678
77333ec71fab [server/utils] convert to py3k using six
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    30
from six.moves import input
77333ec71fab [server/utils] convert to py3k using six
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
    31
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    32
from passlib.utils import handlers as uh, to_hash_str
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    33
from passlib.context import CryptContext
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    34
11091
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    35
from logilab.common.deprecation import deprecated
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    36
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    37
from cubicweb.md5crypt import crypt as md5crypt
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    38
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    39
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    40
class CustomMD5Crypt(uh.HasSalt, uh.GenericHandler):
8398
a9fe30c953be [server/utils] passlib 1.6 is now less tolerant wrt handler names (closes #2349330)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8317
diff changeset
    41
    name = 'cubicwebmd5crypt'
a9fe30c953be [server/utils] passlib 1.6 is now less tolerant wrt handler names (closes #2349330)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8317
diff changeset
    42
    setting_kwds = ('salt',)
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    43
    min_salt_size = 0
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    44
    max_salt_size = 8
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    45
    salt_chars = uh.H64_CHARS
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    46
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    47
    @classmethod
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    48
    def from_string(cls, hash):
8414
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    49
        salt, chk = uh.parse_mc2(hash, u'')
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    50
        if chk is None:
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    51
            raise ValueError('missing checksum')
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    52
        return cls(salt=salt, checksum=chk)
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    53
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    54
    def to_string(self):
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    55
        return to_hash_str(u'%s$%s' % (self.salt, self.checksum or u''))
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    56
8414
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    57
    # passlib 1.5 wants calc_checksum, 1.6 wants _calc_checksum
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    58
    def calc_checksum(self, secret):
8414
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    59
        return md5crypt(secret, self.salt.encode('ascii')).decode('utf-8')
e7243ed7bb0a Fixes for compatibility with passlib 1.6 (closes #2356393)
Julien Cristau <julien.cristau@logilab.fr>
parents: 8398
diff changeset
    60
    _calc_checksum = calc_checksum
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    61
12014
116005b5fce9 [server] Make cubicweb/server/utils.py flake8-clean
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12013
diff changeset
    62
8546
3d2038d6f20d [sources/native] automatically update passwords using deprecated hashes on login
Julien Cristau <julien.cristau@logilab.fr>
parents: 8446
diff changeset
    63
_CRYPTO_CTX = CryptContext(['sha512_crypt', CustomMD5Crypt, 'des_crypt', 'ldap_salted_sha1'],
3d2038d6f20d [sources/native] automatically update passwords using deprecated hashes on login
Julien Cristau <julien.cristau@logilab.fr>
parents: 8446
diff changeset
    64
                           deprecated=['cubicwebmd5crypt', 'des_crypt'])
3d2038d6f20d [sources/native] automatically update passwords using deprecated hashes on login
Julien Cristau <julien.cristau@logilab.fr>
parents: 8446
diff changeset
    65
verify_and_update = _CRYPTO_CTX.verify_and_update
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    66
12014
116005b5fce9 [server] Make cubicweb/server/utils.py flake8-clean
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12013
diff changeset
    67
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    68
def crypt_password(passwd, salt=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    69
    """return the encrypted password using the given salt or a generated one
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    70
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    71
    if salt is None:
11880
bc9d901cb9e6 [server] Use CryptContext's hash method instead of deprecated encrypt method
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11767
diff changeset
    72
        return _CRYPTO_CTX.hash(passwd).encode('ascii')
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    73
    # empty hash, accept any password for backwards compat
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    74
    if salt == '':
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    75
        return salt
8550
63260486de89 [server/utils] catch ValueError from password verification
Julien Cristau <julien.cristau@logilab.fr>
parents: 8546
diff changeset
    76
    try:
63260486de89 [server/utils] catch ValueError from password verification
Julien Cristau <julien.cristau@logilab.fr>
parents: 8546
diff changeset
    77
        if _CRYPTO_CTX.verify(passwd, salt):
63260486de89 [server/utils] catch ValueError from password verification
Julien Cristau <julien.cristau@logilab.fr>
parents: 8546
diff changeset
    78
            return salt
12014
116005b5fce9 [server] Make cubicweb/server/utils.py flake8-clean
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12013
diff changeset
    79
    except ValueError:  # e.g. couldn't identify hash
8550
63260486de89 [server/utils] catch ValueError from password verification
Julien Cristau <julien.cristau@logilab.fr>
parents: 8546
diff changeset
    80
        pass
8317
9c59258e7798 [security] use a stronger encryption algorythm for password, keeping bw compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
    81
    # wrong password
10681
4383f5a30504 [server/utils] crypt_password should always return bytes
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10678
diff changeset
    82
    return b''
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    83
12014
116005b5fce9 [server] Make cubicweb/server/utils.py flake8-clean
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12013
diff changeset
    84
11091
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    85
@deprecated('[3.22] no more necessary, directly get eschema.eid')
9610
29450466273a [server] eschema_eid needs a connection, not a session
Julien Cristau <julien.cristau@logilab.fr>
parents: 9468
diff changeset
    86
def eschema_eid(cnx, eschema):
11091
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    87
    """get eid of the CWEType entity for the given yams type.
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    88
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    89
    This used to be necessary because when the schema has been loaded from the
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    90
    file-system, not from the database, (e.g. during tests), eschema.eid was
29aebc1edd29 [repository] drop usage of no more necessary eschema_eid function
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 10907
diff changeset
    91
    not set.
5066
bf5cbc351e99 [repo] move eschema_eid function from hooks.metadata to server.utils
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4714
diff changeset
    92
    """
bf5cbc351e99 [repo] move eschema_eid function from hooks.metadata to server.utils
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4714
diff changeset
    93
    return eschema.eid
bf5cbc351e99 [repo] move eschema_eid function from hooks.metadata to server.utils
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4714
diff changeset
    94
bf5cbc351e99 [repo] move eschema_eid function from hooks.metadata to server.utils
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4714
diff changeset
    95
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    96
DEFAULT_MSG = 'we need a manager connection on the repository \
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    97
(the server doesn\'t have to run, even should better not)'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    98
12014
116005b5fce9 [server] Make cubicweb/server/utils.py flake8-clean
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12013
diff changeset
    99
1910
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   100
def manager_userpasswd(user=None, msg=DEFAULT_MSG, confirm=False,
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   101
                       passwdmsg='password'):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   102
    if not user:
3701
104b7c326172 check we've some message to display
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3585
diff changeset
   103
        if msg:
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10095
diff changeset
   104
            print(msg)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   105
        while not user:
10678
77333ec71fab [server/utils] convert to py3k using six
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   106
            user = input('login: ')
77333ec71fab [server/utils] convert to py3k using six
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   107
        if PY2:
77333ec71fab [server/utils] convert to py3k using six
Rémi Cardona <remi.cardona@logilab.fr>
parents: 10589
diff changeset
   108
            user = unicode(user, sys.stdin.encoding)
1910
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   109
    passwd = getpass('%s: ' % passwdmsg)
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   110
    if confirm:
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   111
        while True:
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   112
            passwd2 = getpass('confirm password: ')
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   113
            if passwd == passwd2:
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   114
                break
10589
7c23b7de2b8d [py3k] print function
Samuel Trégouët <samuel.tregouet@logilab.fr>
parents: 10095
diff changeset
   115
            print('password doesn\'t match')
1910
864aa3ea0db5 [server] refactor server.utils.manager_userpasswd
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1802
diff changeset
   116
            passwd = getpass('password: ')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   117
    # XXX decode password using stdin encoding then encode it using appl'encoding
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   118
    return user, passwd
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   119
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   120
12011
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   121
if PY2:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   122
    import time  # noqa
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   123
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   124
    class scheduler(sched.scheduler):
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   125
        """Python2 version of sched.scheduler that matches Python3 API."""
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   126
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   127
        def __init__(self, **kwargs):
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   128
            kwargs.setdefault('timefunc', time.time)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   129
            kwargs.setdefault('delayfunc', time.sleep)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   130
            # sched.scheduler is an old-style class.
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   131
            sched.scheduler.__init__(self, **kwargs)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   132
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   133
else:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   134
    scheduler = sched.scheduler
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   135
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   136
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   137
def schedule_periodic_task(scheduler, interval, func, *args):
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   138
    """Enter a task with `func(*args)` as a periodic event in `scheduler`
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   139
    executing at `interval` seconds. Once executed, the task would re-schedule
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   140
    itself unless a BaseException got raised.
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   141
    """
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   142
    @wraps(func)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   143
    def task(*args):
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   144
        restart = True
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   145
        try:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   146
            func(*args)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   147
        except Exception:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   148
            logger = logging.getLogger('cubicweb.scheduler')
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   149
            logger.exception('Unhandled exception in periodic task "%s"',
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   150
                             func.__name__)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   151
        except BaseException as exc:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   152
            logger = logging.getLogger('cubicweb.scheduler')
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   153
            logger.error('periodic task "%s" not re-scheduled due to %r',
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   154
                         func.__name__, exc)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   155
            restart = False
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   156
        finally:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   157
            if restart:
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   158
                scheduler.enter(interval, 1, task, argument=args)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   159
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   160
    return scheduler.enter(interval, 1, task, argument=args)
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   161
d2888fee6031 [server] introduce a scheduler class to run repository "looping tasks"
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 11880
diff changeset
   162
9468
39b7a91a3f4c [repo] pylint cleanup, mainly of imports, with a bit of style
Julien Cristau <julien.cristau@logilab.fr>
parents: 9467
diff changeset
   163
_MARKER = object()
12014
116005b5fce9 [server] Make cubicweb/server/utils.py flake8-clean
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12013
diff changeset
   164
116005b5fce9 [server] Make cubicweb/server/utils.py flake8-clean
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 12013
diff changeset
   165
6381
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   166
def func_name(func):
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   167
    name = getattr(func, '__name__', _MARKER)
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   168
    if name is _MARKER:
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   169
        name = getattr(func, 'func_name', _MARKER)
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   170
    if name is _MARKER:
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   171
        name = repr(func)
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   172
    return name
2105
92ea410806fe refactor sources configuration, add source to sources when using a cube defining
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1977
diff changeset
   173
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   174
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   175
class RepoThread(Thread):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   176
    """subclass of thread so it auto remove itself from a given list once
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   177
    executed
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   178
    """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   179
    def __init__(self, target, running_threads):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   180
        def auto_remove_func(self=self, func=target):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   181
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   182
                func()
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7573
diff changeset
   183
            except Exception:
6765
b922e3a817e9 fix ticket #1382716 (problem was actually more subtle than I originally thought)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6427
diff changeset
   184
                logger = logging.getLogger('cubicweb.repository')
b922e3a817e9 fix ticket #1382716 (problem was actually more subtle than I originally thought)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6427
diff changeset
   185
                logger.exception('Unhandled exception in RepoThread %s', self._name)
b922e3a817e9 fix ticket #1382716 (problem was actually more subtle than I originally thought)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6427
diff changeset
   186
                raise
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   187
            finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   188
                self.running_threads.remove(self)
1138
22f634977c95 make pylint happy, fix some bugs on the way
sylvain.thenault@logilab.fr
parents: 1134
diff changeset
   189
        Thread.__init__(self, target=auto_remove_func)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   190
        self.running_threads = running_threads
6381
c9eed5037223 [repo threads] Add several safety when looking for a callable name.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 6128
diff changeset
   191
        self._name = func_name(target)
1802
d628defebc17 delete-trailing-whitespace + some copyright update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 1138
diff changeset
   192
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   193
    def start(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   194
        self.running_threads.append(self)
3585
cd437d24aa65 use daemon thread
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2708
diff changeset
   195
        self.daemon = True
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   196
        Thread.start(self)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   197
5376
2c3f14bc2590 [python2.6] don't add a name property on Thread
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5066
diff changeset
   198
    def getName(self):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   199
        return '%s(%s)' % (self._name, Thread.getName(self))