server/repository.py
author Alexandre Fayolle <alexandre.fayolle@logilab.fr>
Wed, 16 Mar 2011 09:37:46 +0100
brancholdstable
changeset 7083 b8e35cde46e9
parent 7064 b8ce2266d026
child 7085 36e91d19188b
permissions -rw-r--r--
help pylint by explicitely defining some attributes * the logging functions automatically set by set_log_methods * the attributes of Hook classes (eidfrom, entity...):
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     1
# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
"""Defines the central class for the CubicWeb RQL server: the repository.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
The repository is an abstraction allowing execution of rql queries against
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
data sources. Most of the work is actually done in helper classes. The
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
repository mainly:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
* brings these classes all together to provide a single access
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2473
diff changeset
    25
  point to a cubicweb instance.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
* handles session management
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
* provides method for pyro registration, to call if pyro is enabled
5587
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
    28
"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    29
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
    30
from __future__ import with_statement
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
    31
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    32
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    33
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    34
import sys
5587
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
    35
import threading
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    36
import Queue
6390
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
    37
from itertools import chain
4719
aaed3f813ef8 kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4714
diff changeset
    38
from os.path import join
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 636
diff changeset
    39
from datetime import datetime
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    40
from time import time, localtime, strftime
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    41
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
    42
from logilab.common.decorators import cached, clear_cache
3757
122a01751d59 2.4 compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3694
diff changeset
    43
from logilab.common.compat import any
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
    44
from logilab.common import flatten
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    45
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    46
from yams import BadSchemaDefinition
5030
5238d9a8dfee [form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4984
diff changeset
    47
from yams.schema import role_name
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    48
from rql import RQLSyntaxError
6390
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
    49
from rql.utils import rqlvar_maker
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
    51
from cubicweb import (CW_SOFTWARE_ROOT, CW_MIGRATION_MAP, QueryError,
2709
6ee816eb9f25 [hooksmanager] hooks should be reseted after vreg reload
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2708
diff changeset
    52
                      UnknownEid, AuthenticationError, ExecutionError,
3042
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
    53
                      ETypeNotSupportedBySources, MultiSourcesError,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    54
                      BadConnectionId, Unauthorized, ValidationError,
6211
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
    55
                      RepositoryError, UniqueTogetherError, typed_eid, onevent)
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
    56
from cubicweb import cwvreg, schema, server
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
    57
from cubicweb.server import utils, hook, pool, querier, sources
5079
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
    58
from cubicweb.server.session import Session, InternalSession, InternalManager, \
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
    59
     security_enabled
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
    60
from cubicweb.server.ssplanner import EditedEntity
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    61
6467
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    62
def prefill_entity_caches(entity, relations):
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    63
    session = entity._cw
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    64
    # prefill entity relation caches
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    65
    for rschema in entity.e_schema.subject_relations():
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    66
        rtype = str(rschema)
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    67
        if rtype in schema.VIRTUAL_RTYPES:
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    68
            continue
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    69
        if rschema.final:
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    70
            entity.cw_attr_cache.setdefault(rtype, None)
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    71
        else:
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    72
            entity.cw_set_relation_cache(rtype, 'subject',
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    73
                                         session.empty_rset())
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    74
    for rschema in entity.e_schema.object_relations():
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    75
        rtype = str(rschema)
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    76
        if rtype in schema.VIRTUAL_RTYPES:
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    77
            continue
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    78
        entity.cw_set_relation_cache(rtype, 'object', session.empty_rset())
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    79
    # set inlined relation cache before call to after_add_entity
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    80
    for attr, value in relations:
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    81
        session.update_rel_cache_add(entity.eid, attr, value)
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    82
        del_existing_rel_if_needed(session, entity.eid, attr, value)
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    83
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    84
def del_existing_rel_if_needed(session, eidfrom, rtype, eidto):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    85
    """delete existing relation when adding a new one if card is 1 or ?
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
    have to be done once the new relation has been inserted to avoid having
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    88
    an entity without a relation for some time
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    89
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
    this kind of behaviour has to be done in the repository so we don't have
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
    hooks order hazardness
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
    """
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
    93
    # skip that for internal session or if integrity explicitly disabled
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
    94
    #
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
    95
    # XXX we should imo rely on the orm to first fetch existing entity if any
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
    96
    # then delete it.
4756
a2c73b272c9b do not check for existant entity when relation of single card is added but integrity hooks are deactivated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
    97
    if session.is_internal_session \
5090
8c39d2bf58fd [repo creation] removing existing entities of 'single' cardinality relatino should be considered as 'activeintegrity' hook. Also don't disable that category during repo creation to avoid pb such as two default workflows for one entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5084
diff changeset
    98
           or not session.is_hook_category_activated('activeintegrity'):
4687
082e66184f71 [integrity] skip auto-delete for internal sessions (but *not* for super session, see 4642:921737d2e3a8)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4686
diff changeset
    99
        return
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   100
    card = session.schema_rproperty(rtype, eidfrom, eidto, 'cardinality')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   101
    # one may be tented to check for neweids but this may cause more than one
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   102
    # relation even with '1?'  cardinality if thoses relations are added in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   103
    # same transaction where the entity is being created. This never occurs from
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   104
    # the web interface but may occurs during test or dbapi connection (though
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   105
    # not expected for this).  So: don't do it, we pretend to ensure repository
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   106
    # consistency.
4673
6f8b925a29f4 auto-delete relation with single cardinality using execute, not unsafe_execute, so permissions are correctly checked
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4643
diff changeset
   107
    #
6189
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   108
    # notes:
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   109
    # * inlined relations will be implicitly deleted for the subject entity
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   110
    # * we don't want read permissions to be applied but we want delete
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   111
    #   permission to be checked
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   112
    if card[0] in '1?' and not session.repo.schema.rschema(rtype).inlined:
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   113
        with security_enabled(session, read=False):
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   114
            session.execute('DELETE X %s Y WHERE X eid %%(x)s, '
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   115
                            'NOT Y eid %%(y)s' % rtype,
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
   116
                                {'x': eidfrom, 'y': eidto})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   117
    if card[1] in '1?':
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   118
        with security_enabled(session, read=False):
5711
ea2538a3c622 [repo] fix typo
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5641
diff changeset
   119
            session.execute('DELETE X %s Y WHERE Y eid %%(y)s, '
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   120
                            'NOT X eid %%(x)s' % rtype,
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
   121
                            {'x': eidfrom, 'y': eidto})
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   122
3694
33dbb1da1db9 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   123
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   124
class Repository(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   125
    """a repository provides access to a set of persistent storages for
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   126
    entities and relations
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   127
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   128
    XXX protect pyro access
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
    """
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   130
5442
3ed8afbbdf70 [webconfig] refactor/cleanup debug mode management on startup: simply use config.debugmode instead of debug argument everywhere...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   131
    def __init__(self, config, vreg=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   132
        self.config = config
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   133
        if vreg is None:
5442
3ed8afbbdf70 [webconfig] refactor/cleanup debug mode management on startup: simply use config.debugmode instead of debug argument everywhere...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   134
            vreg = cwvreg.CubicWebVRegistry(config)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   135
        self.vreg = vreg
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
        self.pyro_registered = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
        self.info('starting repository from %s', self.config.apphome)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   138
        # dictionary of opened sessions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   139
        self._sessions = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   140
        # list of functions to be called at regular interval
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
        self._looping_tasks = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
        # list of running threads
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   143
        self._running_threads = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   144
        # initial schema, should be build or replaced latter
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   145
        self.schema = schema.CubicWebSchema(config.appid)
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3042
diff changeset
   146
        self.vreg.schema = self.schema # until actual schema is loaded...
5906
d40ced753291 [repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5813
diff changeset
   147
        # shutdown flag
d40ced753291 [repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5813
diff changeset
   148
        self.shutting_down = False
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   149
        # sources (additional sources info in the system database)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   150
        self.system_source = self.get_source('native', 'system',
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   151
                                             config.sources()['system'])
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   152
        self.sources = [self.system_source]
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   153
        self.sources_by_uri = {'system': self.system_source}
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   154
        # querier helper, need to be created after sources initialization
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   155
        self.querier = querier.QuerierHelper(self, self.schema)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   156
        # cache eid -> type / source
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   157
        self._type_source_cache = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   158
        # cache (extid, source uri) -> eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   159
        self._extid_cache = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
        # open some connections pools
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2929
diff changeset
   161
        if config.open_connections_pools:
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2929
diff changeset
   162
            self.open_connections_pools()
5079
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   163
        @onevent('after-registry-reload', self)
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   164
        def fix_user_classes(self):
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   165
            usercls = self.vreg['etypes'].etype_class('CWUser')
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   166
            for session in self._sessions.values():
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   167
                if not isinstance(session.user, InternalManager):
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   168
                    session.user.__class__ = usercls
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2929
diff changeset
   169
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2929
diff changeset
   170
    def open_connections_pools(self):
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2929
diff changeset
   171
        config = self.config
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   172
        self._available_pools = Queue.Queue()
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   173
        self._available_pools.put_nowait(pool.ConnectionsPool(self.sources))
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   174
        if config.quick_start:
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   175
            # quick start, usually only to get a minimal repository to get cubes
5084
d8f491cb046c [repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5079
diff changeset
   176
            # information (eg dump/restore/...)
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   177
            config._cubes = ()
5084
d8f491cb046c [repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5079
diff changeset
   178
            # only load hooks and entity classes in the registry
d8f491cb046c [repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5079
diff changeset
   179
            config.cube_appobject_path = set(('hooks', 'entities'))
d8f491cb046c [repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5079
diff changeset
   180
            config.cubicweb_appobject_path = set(('hooks', 'entities'))
d8f491cb046c [repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5079
diff changeset
   181
            self.set_schema(config.load_schema())
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   182
            config['connections-pool-size'] = 1
5084
d8f491cb046c [repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5079
diff changeset
   183
            # will be reinitialized later from cubes found in the database
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   184
            config._cubes = None
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   185
        elif config.creating:
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   186
            # repository creation
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   187
            config.bootstrap_cubes()
2968
0e3460341023 somewhat painful backport of 3.5 branch, should mostly be ok
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2893 2967
diff changeset
   188
            self.set_schema(config.load_schema(), resetvreg=False)
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   189
            # need to load the Any and CWUser entity types
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   190
            etdirectory = join(CW_SOFTWARE_ROOT, 'entities')
1317
sylvain.thenault@logilab.fr
parents: 1316
diff changeset
   191
            self.vreg.init_registration([etdirectory])
3409
c684606a95e0 [bootstrap] metadata hooks need to be registered at bootstrap time for db creation
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3381
diff changeset
   192
            for modname in ('__init__', 'authobjs', 'wfobjs'):
c684606a95e0 [bootstrap] metadata hooks need to be registered at bootstrap time for db creation
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3381
diff changeset
   193
                self.vreg.load_file(join(etdirectory, '%s.py' % modname),
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   194
                                    'cubicweb.entities.%s' % modname)
5447
f29dafba250a [vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5442
diff changeset
   195
            hooksdirectory = join(CW_SOFTWARE_ROOT, 'hooks')
f29dafba250a [vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5442
diff changeset
   196
            self.vreg.load_file(join(hooksdirectory, 'metadata.py'),
f29dafba250a [vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5442
diff changeset
   197
                                'cubicweb.hooks.metadata')
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   198
        elif config.read_instance_schema:
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   199
            # normal start: load the instance schema from the database
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   200
            self.fill_schema()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   201
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   202
            # test start: use the file system schema (quicker)
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2473
diff changeset
   203
            self.warning("set fs instance'schema")
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   204
            config.bootstrap_cubes()
2961
6d833a440575 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   205
            self.set_schema(config.load_schema())
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   206
        if not config.creating:
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   207
            self.init_sources_from_database()
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   208
            if 'CWProperty' in self.schema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   209
                self.vreg.init_properties(self.properties())
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   210
            # call source's init method to complete their initialisation if
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   211
            # needed (for instance looking for persistent configuration using an
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   212
            # internal session, which is not possible until pools have been
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   213
            # initialized)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   214
            for source in self.sources:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   215
                source.init()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   216
        else:
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
   217
            # call init_creating so that for instance native source can
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
   218
            # configurate tsearch according to postgres version
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   219
            for source in self.sources:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   220
                source.init_creating()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   221
        # close initialization pool and reopen fresh ones for proper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   222
        # initialization now that we know cubes
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   223
        self._get_pool().close(True)
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   224
        # list of available pools (we can't iterate on Queue instance)
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   225
        self.pools = []
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   226
        for i in xrange(config['connections-pool-size']):
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   227
            self.pools.append(pool.ConnectionsPool(self.sources))
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   228
            self._available_pools.put_nowait(self.pools[-1])
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   229
        if config.quick_start:
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   230
            config.init_cubes(self.get_cubes())
5093
8d073d2e089d [optimization] improve massive write performance by optimizing hooks selection
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5090
diff changeset
   231
        self.hm = hook.HooksManager(self.vreg)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   232
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   233
    # internals ###############################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   234
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   235
    def init_sources_from_database(self):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   236
        self.sources_by_eid = {}
6664
7b6acf314a7a [repo] don't attempt to read sources from database on quick start. Fix crash after restoring a pre 3.10 database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6629
diff changeset
   237
        if self.config.quick_start \
7b6acf314a7a [repo] don't attempt to read sources from database on quick start. Fix crash after restoring a pre 3.10 database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6629
diff changeset
   238
               or not 'CWSource' in self.schema: # # 3.10 migration
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   239
            return
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   240
        session = self.internal_session()
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   241
        try:
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   242
            # FIXME: sources should be ordered (add_entity priority)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   243
            for sourceent in session.execute(
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   244
                'Any S, SN, SA, SC WHERE S is CWSource, '
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   245
                'S name SN, S type SA, S config SC').entities():
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   246
                if sourceent.name == 'system':
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   247
                    self.system_source.eid = sourceent.eid
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   248
                    self.sources_by_eid[sourceent.eid] = self.system_source
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   249
                    continue
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   250
                self.add_source(sourceent, add_to_pools=False)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   251
        finally:
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   252
            session.close()
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   253
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   254
    def _clear_planning_caches(self):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   255
        for cache in ('source_defs', 'is_multi_sources_relation',
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   256
                      'can_cross_relation', 'rel_type_sources'):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   257
            clear_cache(self, cache)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   258
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   259
    def add_source(self, sourceent, add_to_pools=True):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   260
        source = self.get_source(sourceent.type, sourceent.name,
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   261
                                 sourceent.host_config)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   262
        source.eid = sourceent.eid
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   263
        self.sources_by_eid[sourceent.eid] = source
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   264
        self.sources_by_uri[sourceent.name] = source
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   265
        if self.config.source_enabled(source):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   266
            self.sources.append(source)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   267
            self.querier.set_planner()
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   268
            if add_to_pools:
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   269
                for pool in self.pools:
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   270
                    pool.add_source(source)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   271
        self._clear_planning_caches()
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   272
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   273
    def remove_source(self, uri):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   274
        source = self.sources_by_uri.pop(uri)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   275
        del self.sources_by_eid[source.eid]
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   276
        if self.config.source_enabled(source):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   277
            self.sources.remove(source)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   278
            self.querier.set_planner()
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   279
            for pool in self.pools:
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   280
                pool.remove_source(source)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   281
        self._clear_planning_caches()
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   282
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   283
    def get_source(self, type, uri, source_config):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   284
        # set uri and type in source config so it's available through
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   285
        # source_defs()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   286
        source_config['uri'] = uri
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   287
        source_config['type'] = type
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   288
        return sources.get_source(type, source_config, self)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   289
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2961
diff changeset
   290
    def set_schema(self, schema, resetvreg=True, rebuildinfered=True):
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2961
diff changeset
   291
        if rebuildinfered:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2961
diff changeset
   292
            schema.rebuild_infered_relations()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   293
        self.info('set schema %s %#x', schema.name, id(schema))
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3042
diff changeset
   294
        if resetvreg:
2881
d1a5b77e42bc fix set_schema/init_cubes order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2880
diff changeset
   295
            if self.config._cubes is None:
d1a5b77e42bc fix set_schema/init_cubes order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2880
diff changeset
   296
                self.config.init_cubes(self.get_cubes())
5447
f29dafba250a [vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5442
diff changeset
   297
            # trigger full reload of all appobjects
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3042
diff changeset
   298
            self.vreg.set_schema(schema)
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3042
diff changeset
   299
        else:
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3042
diff changeset
   300
            self.vreg._set_schema(schema)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   301
        self.querier.set_schema(schema)
6126
aca6a2c357fd [repository] enabled sources refactoring: to avoid error, we should always have all known source in repo.sources_by_uri and only enabled ones in repo.sources, so we still have access to the definition of temporarily disabled sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6061
diff changeset
   302
        # don't use self.sources, we may want to give schema even to disabled
aca6a2c357fd [repository] enabled sources refactoring: to avoid error, we should always have all known source in repo.sources_by_uri and only enabled ones in repo.sources, so we still have access to the definition of temporarily disabled sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6061
diff changeset
   303
        # sources
aca6a2c357fd [repository] enabled sources refactoring: to avoid error, we should always have all known source in repo.sources_by_uri and only enabled ones in repo.sources, so we still have access to the definition of temporarily disabled sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6061
diff changeset
   304
        for source in self.sources_by_uri.values():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   305
            source.set_schema(schema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   306
        self.schema = schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   307
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   308
    def fill_schema(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   309
        """lod schema from the repository"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   310
        from cubicweb.server.schemaserial import deserialize_schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   311
        self.info('loading schema from the repository')
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   312
        appschema = schema.CubicWebSchema(self.config.appid)
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
   313
        self.set_schema(self.config.load_bootstrap_schema(), resetvreg=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
        self.debug('deserializing db schema into %s %#x', appschema.name, id(appschema))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   315
        session = self.internal_session()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   316
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   317
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   318
                deserialize_schema(appschema, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   319
            except BadSchemaDefinition:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   320
                raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
            except Exception, ex:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   322
                import traceback
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   323
                traceback.print_exc()
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   324
                raise Exception('Is the database initialised ? (cause: %s)' %
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
                                (ex.args and ex.args[0].strip() or 'unknown')), \
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   326
                                None, sys.exc_info()[-1]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   327
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   328
            session.close()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
        self.set_schema(appschema)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   330
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
    def start_looping_tasks(self):
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   332
        if not (self.config.creating or self.config.repairing
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   333
                or self.config.quick_start):
4958
665eacdd8c50 [repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   334
            # call instance level initialisation hooks
665eacdd8c50 [repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   335
            self.hm.call_hooks('server_startup', repo=self)
665eacdd8c50 [repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   336
            # register a task to cleanup expired session
6012
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   337
            self.cleanup_session_time = self.config['cleanup-session-time'] or 60 * 60 * 24
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   338
            assert self.cleanup_session_time > 0
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   339
            cleanup_session_interval = min(60*60, self.cleanup_session_time / 3)
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   340
            self.looping_task(cleanup_session_interval, self.clean_sessions)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   341
        assert isinstance(self._looping_tasks, list), 'already started'
2708
60d728bdcba5 allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2667
diff changeset
   342
        for i, (interval, func, args) in enumerate(self._looping_tasks):
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   343
            self._looping_tasks[i] = task = utils.LoopTask(interval, func, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   344
            self.info('starting task %s with interval %.2fs', task.name,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
                      interval)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   346
            task.start()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   347
        # ensure no tasks will be further added
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   348
        self._looping_tasks = tuple(self._looping_tasks)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
2708
60d728bdcba5 allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2667
diff changeset
   350
    def looping_task(self, interval, func, *args):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   351
        """register a function to be called every `interval` seconds.
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   352
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   353
        looping tasks can only be registered during repository initialization,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   354
        once done this method will fail.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   355
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   356
        try:
2708
60d728bdcba5 allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2667
diff changeset
   357
            self._looping_tasks.append( (interval, func, args) )
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   358
        except AttributeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   359
            raise RuntimeError("can't add looping task once the repository is started")
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   360
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   361
    def threaded_task(self, func):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   362
        """start function in a separated thread"""
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   363
        t = utils.RepoThread(func, self._running_threads)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   364
        t.start()
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   365
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   366
    #@locked
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   367
    def _get_pool(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   368
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   369
            return self._available_pools.get(True, timeout=5)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   370
        except Queue.Empty:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   371
            raise Exception('no pool available after 5 secs, probably either a '
4706
6035e96b64dd added stats for munin collecting #615844 - from 027bbff3659f
arthur
parents: 4689
diff changeset
   372
                            'bug in code (too many uncommited/rollbacked '
6035e96b64dd added stats for munin collecting #615844 - from 027bbff3659f
arthur
parents: 4689
diff changeset
   373
                            'connections) or too much load on the server (in '
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   374
                            'which case you can try to set a bigger '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   375
                            'connections pools size)')
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   376
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   377
    def _free_pool(self, pool):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   378
        self._available_pools.put_nowait(pool)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   379
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   380
    def pinfo(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   381
        # XXX: session.pool is accessed from a local storage, would be interesting
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   382
        #      to see if there is a pool set in any thread specific data)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   383
        return '%s: %s (%s)' % (self._available_pools.qsize(),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   384
                                ','.join(session.user.login for session in self._sessions.values()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
                                         if session.pool),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   386
                                threading.currentThread())
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   387
    def shutdown(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   388
        """called on server stop event to properly close opened sessions and
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   389
        connections
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   390
        """
5906
d40ced753291 [repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5813
diff changeset
   391
        assert not self.shutting_down, 'already shutting down'
d40ced753291 [repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5813
diff changeset
   392
        self.shutting_down = True
5749
b4393b681f7a [repo] on repository shutdown, we've to close the new eid creation connection
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5747
diff changeset
   393
        self.system_source.shutdown()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   394
        if isinstance(self._looping_tasks, tuple): # if tasks have been started
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   395
            for looptask in self._looping_tasks:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   396
                self.info('canceling task %s...', looptask.name)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   397
                looptask.cancel()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   398
                looptask.join()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   399
                self.info('task %s finished', looptask.name)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   400
        for thread in self._running_threads:
5376
2c3f14bc2590 [python2.6] don't add a name property on Thread
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5292
diff changeset
   401
            self.info('waiting thread %s...', thread.getName())
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   402
            thread.join()
5376
2c3f14bc2590 [python2.6] don't add a name property on Thread
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5292
diff changeset
   403
            self.info('thread %s finished', thread.getName())
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   404
        if not (self.config.creating or self.config.repairing
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   405
                or self.config.quick_start):
4946
2654cd5c87e0 [repo] don't call server_shutdown hooks when creating/repairing instance to be consistence with server_startup hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4862
diff changeset
   406
            self.hm.call_hooks('server_shutdown', repo=self)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   407
        self.close_sessions()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   408
        while not self._available_pools.empty():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   409
            pool = self._available_pools.get_nowait()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   410
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   411
                pool.close(True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   412
            except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   413
                self.exception('error while closing %s' % pool)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   414
                continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   415
        if self.pyro_registered:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   416
            pyro_unregister(self.config)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
        hits, misses = self.querier.cache_hit, self.querier.cache_miss
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   418
        try:
5747
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   419
            self.info('rql st cache hit/miss: %s/%s (%s%% hits)', hits, misses,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   420
                      (hits * 100) / (hits + misses))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   421
            hits, misses = self.system_source.cache_hit, self.system_source.cache_miss
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   422
            self.info('sql cache hit/miss: %s/%s (%s%% hits)', hits, misses,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
                      (hits * 100) / (hits + misses))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   424
            nocache  = self.system_source.no_cache
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   425
            self.info('sql cache usage: %s/%s (%s%%)', hits+ misses, nocache,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   426
                      ((hits + misses) * 100) / (hits + misses + nocache))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   427
        except ZeroDivisionError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   428
            pass
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   429
2267
e1d2df3f1091 move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2245
diff changeset
   430
    def _login_from_email(self, login):
e1d2df3f1091 move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2245
diff changeset
   431
        session = self.internal_session()
e1d2df3f1091 move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2245
diff changeset
   432
        try:
e1d2df3f1091 move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2245
diff changeset
   433
            rset = session.execute('Any L WHERE U login L, U primary_email M, '
5073
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   434
                                   'M address %(login)s', {'login': login},
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   435
                                   build_descr=False)
2267
e1d2df3f1091 move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2245
diff changeset
   436
            if rset.rowcount == 1:
e1d2df3f1091 move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2245
diff changeset
   437
                login = rset[0][0]
e1d2df3f1091 move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2245
diff changeset
   438
        finally:
e1d2df3f1091 move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2245
diff changeset
   439
            session.close()
e1d2df3f1091 move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2245
diff changeset
   440
        return login
e1d2df3f1091 move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2245
diff changeset
   441
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3629
diff changeset
   442
    def authenticate_user(self, session, login, **kwargs):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   443
        """validate login / password, raise AuthenticationError on failure
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   444
        return associated CWUser instance on success
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   445
        """
2267
e1d2df3f1091 move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2245
diff changeset
   446
        if self.vreg.config['allow-email-login'] and '@' in login:
e1d2df3f1091 move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2245
diff changeset
   447
            login = self._login_from_email(login)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   448
        for source in self.sources:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   449
            if source.support_entity('CWUser'):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   450
                try:
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3629
diff changeset
   451
                    eid = source.authenticate(session, login, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   452
                    break
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   453
                except AuthenticationError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   454
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   455
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   456
            raise AuthenticationError('authentication failed with all sources')
2268
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   457
        cwuser = self._build_user(session, eid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   458
        if self.config.consider_user_state and \
5556
9ab2b4c74baf [entity] introduce a new 'adapters' registry
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5447
diff changeset
   459
               not cwuser.cw_adapt_to('IWorkflowable').state in cwuser.AUTHENTICABLE_STATES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   460
            raise AuthenticationError('user is not in authenticable state')
2268
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   461
        return cwuser
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   462
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   463
    def _build_user(self, session, eid):
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   464
        """return a CWUser entity for user with the given eid"""
2650
18aec79ec3a3 R [vreg] important refactoring of the vregistry, moving behaviour to end dictionnary (and so leaving room for more flexibility ; keep bw compat ; update api usage in cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2647
diff changeset
   465
        cls = self.vreg['etypes'].etype_class('CWUser')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   466
        rql = cls.fetch_rql(session.user, ['X eid %(x)s'])
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
   467
        rset = session.execute(rql, {'x': eid})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   468
        assert len(rset) == 1, rset
2268
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   469
        cwuser = rset.get_entity(0, 0)
6491
ee9a10b6620e pylint option update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6467
diff changeset
   470
        # pylint: disable=W0104
2268
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   471
        # prefetch / cache cwuser's groups and properties. This is especially
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   472
        # useful for internal sessions to avoid security insertions
2268
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   473
        cwuser.groups
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   474
        cwuser.properties
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   475
        return cwuser
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   476
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   477
    # public (dbapi) interface ################################################
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   478
5587
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   479
    def stats(self): # XXX restrict to managers session?
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   480
        """Return a dictionary containing some statistics about the repository
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   481
        resources usage.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   482
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   483
        This is a public method, not requiring a session id.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   484
        """
5587
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   485
        results = {}
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   486
        querier = self.querier
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   487
        source = self.system_source
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   488
        for size, maxsize, hits, misses, title in (
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   489
            (len(querier._rql_cache), self.config['rql-cache-size'],
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   490
            querier.cache_hit, querier.cache_miss, 'rqlt_st'),
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   491
            (len(source._cache), self.config['rql-cache-size'],
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   492
            source.cache_hit, source.cache_miss, 'sql'),
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   493
            ):
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   494
            results['%s_cache_size' % title] =  '%s / %s' % (size, maxsize)
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   495
            results['%s_cache_hit' % title] =  hits
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   496
            results['%s_cache_miss' % title] = misses
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   497
            results['%s_cache_hit_percent' % title] = (hits * 100) / (hits + misses)
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   498
        results['sql_no_cache'] = self.system_source.no_cache
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   499
        results['nb_open_sessions'] = len(self._sessions)
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   500
        results['nb_active_threads'] = threading.activeCount()
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   501
        results['looping_tasks'] = ', '.join(str(t) for t in self._looping_tasks)
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   502
        results['available_pools'] = self._available_pools.qsize()
5606
61b28589d33f merge back to stable some changes made on site for a customer.
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5587 5603
diff changeset
   503
        results['threads'] = ', '.join(sorted(str(t) for t in threading.enumerate()))
5587
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   504
        return results
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   505
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   506
    def get_schema(self):
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   507
        """Return the instance schema.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   508
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   509
        This is a public method, not requiring a session id.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   510
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   511
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   512
            # necessary to support pickling used by pyro
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   513
            self.schema.__hashmode__ = 'pickle'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   514
            return self.schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   515
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   516
            self.schema.__hashmode__ = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   517
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   518
    def get_cubes(self):
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   519
        """Return the list of cubes used by this instance.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   520
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   521
        This is a public method, not requiring a session id.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   522
        """
2473
490f88fb99b6 new distinguish repairing/creating from regular start.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2268
diff changeset
   523
        versions = self.get_versions(not (self.config.creating
4689
4eb1f4490538 [test] skipping versions checking during test is enough, no need for monkey patch
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4687
diff changeset
   524
                                          or self.config.repairing
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   525
                                          or self.config.quick_start
4689
4eb1f4490538 [test] skipping versions checking during test is enough, no need for monkey patch
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4687
diff changeset
   526
                                          or self.config.mode == 'test'))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   527
        cubes = list(versions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   528
        cubes.remove('cubicweb')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   529
        return cubes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   530
6308
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   531
    def get_option_value(self, option, foreid=None):
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   532
        """Return the value for `option` in the configuration. If `foreid` is
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   533
        specified, the actual repository to which this entity belongs is
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   534
        derefenced and the option value retrieved from it.
6257
7eb5f1aed728 [repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6256
diff changeset
   535
7eb5f1aed728 [repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6256
diff changeset
   536
        This is a public method, not requiring a session id.
7eb5f1aed728 [repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6256
diff changeset
   537
        """
7eb5f1aed728 [repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6256
diff changeset
   538
        # XXX we may want to check we don't give sensible information
6308
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   539
        if foreid is None:
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   540
            return self.config[option]
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   541
        _, sourceuri, extid = self.type_and_source_from_eid(foreid)
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   542
        if sourceuri == 'system':
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   543
            return self.config[option]
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   544
        pool = self._get_pool()
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   545
        try:
7064
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   546
            cnx = pool.connection(sourceuri)
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   547
            # needed to check connection is valid and usable by the current
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   548
            # thread
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   549
            newcnx = self.sources_by_uri[sourceuri].check_connection(cnx)
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   550
            if newcnx is not None:
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   551
                cnx = newcnx
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   552
            return cnx.get_option_value(option, extid)
6308
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   553
        finally:
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   554
            self._free_pool(pool)
6257
7eb5f1aed728 [repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6256
diff changeset
   555
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   556
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   557
    def get_versions(self, checkversions=False):
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   558
        """Return the a dictionary containing cubes used by this instance
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   559
        as key with their version as value, including cubicweb version.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   560
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   561
        This is a public method, not requiring a session id.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   562
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   563
        from logilab.common.changelog import Version
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   564
        vcconf = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   565
        session = self.internal_session()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   566
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   567
            for pk, version in session.execute(
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   568
                'Any K,V WHERE P is CWProperty, P value V, P pkey K, '
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   569
                'P pkey ~="system.version.%"', build_descr=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   570
                cube = pk.split('.')[-1]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   571
                # XXX cubicweb migration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   572
                if cube in CW_MIGRATION_MAP:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   573
                    cube = CW_MIGRATION_MAP[cube]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   574
                version = Version(version)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   575
                vcconf[cube] = version
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   576
                if checkversions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   577
                    if cube != 'cubicweb':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   578
                        fsversion = self.config.cube_version(cube)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   579
                    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   580
                        fsversion = self.config.cubicweb_version()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   581
                    if version < fsversion:
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2473
diff changeset
   582
                        msg = ('instance has %s version %s but %s '
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   583
                               'is installed. Run "cubicweb-ctl upgrade".')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   584
                        raise ExecutionError(msg % (cube, version, fsversion))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   585
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   586
            session.close()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   587
        return vcconf
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   588
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   589
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   590
    def source_defs(self):
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   591
        """Return the a dictionary containing source uris as value and a
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   592
        dictionary describing each source as value.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   593
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   594
        This is a public method, not requiring a session id.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   595
        """
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   596
        sources = {}
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   597
        # remove sensitive information
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   598
        for uri, source in self.sources_by_uri.iteritems():
6722
3341521d857b [repo source] rename attribute so it's much easier to grasp its role
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6674
diff changeset
   599
            sources[uri] = source.public_config
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   600
        return sources
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   601
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   602
    def properties(self):
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   603
        """Return a result set containing system wide properties.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   604
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   605
        This is a public method, not requiring a session id.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   606
        """
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   607
        session = self.internal_session()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   608
        try:
5747
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   609
            # don't use session.execute, we don't want rset.req set
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   610
            return self.querier.execute(session, 'Any K,V WHERE P is CWProperty,'
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   611
                                        'P pkey K, P value V, NOT P for_user U',
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   612
                                        build_descr=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   613
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   614
            session.close()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   615
5032
e3fa27fc0d9a cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5030
diff changeset
   616
    # XXX protect this method: anonymous should be allowed and registration
e3fa27fc0d9a cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5030
diff changeset
   617
    # plugged
1372
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   618
    def register_user(self, login, password, email=None, **kwargs):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   619
        """check a user with the given login exists, if not create it with the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   620
        given password. This method is designed to be used for anonymous
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   621
        registration on public web site.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   622
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   623
        session = self.internal_session()
1372
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   624
        # for consistency, keep same error as unique check hook (although not required)
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   625
        errmsg = session._('the value "%s" is already used, use another one')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   626
        try:
5073
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   627
            if (session.execute('CWUser X WHERE X login %(login)s', {'login': login},
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   628
                                build_descr=False)
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   629
                or session.execute('CWUser X WHERE X use_email C, C address %(login)s',
5073
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   630
                                   {'login': login}, build_descr=False)):
5030
5238d9a8dfee [form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4984
diff changeset
   631
                qname = role_name('login', 'subject')
5238d9a8dfee [form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4984
diff changeset
   632
                raise ValidationError(None, {qname: errmsg % login})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   633
            # we have to create the user
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
   634
            user = self.vreg['etypes'].etype_class('CWUser')(session)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   635
            if isinstance(password, unicode):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   636
                # password should *always* be utf8 encoded
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   637
                password = password.encode('UTF8')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   638
            kwargs['login'] = login
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   639
            kwargs['upassword'] = password
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
   640
            self.glob_add_entity(session, EditedEntity(user, **kwargs))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   641
            session.execute('SET X in_group G WHERE X eid %(x)s, G name "users"',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   642
                            {'x': user.eid})
1372
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   643
            if email or '@' in login:
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   644
                d = {'login': login, 'email': email or login}
5073
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   645
                if session.execute('EmailAddress X WHERE X address %(email)s', d,
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   646
                                   build_descr=False):
5030
5238d9a8dfee [form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4984
diff changeset
   647
                    qname = role_name('address', 'subject')
5238d9a8dfee [form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4984
diff changeset
   648
                    raise ValidationError(None, {qname: errmsg % d['email']})
1372
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   649
                session.execute('INSERT EmailAddress X: X address %(email)s, '
5032
e3fa27fc0d9a cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5030
diff changeset
   650
                                'U primary_email X, U use_email X '
5073
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   651
                                'WHERE U login %(login)s', d, build_descr=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   652
            session.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   653
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   654
            session.close()
594
76218d42d21f return success or not on creation of user
Arthur Lutz <arthur.lutz@logilab.fr>
parents: 479
diff changeset
   655
        return True
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   656
6390
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   657
    def find_users(self, fetch_attrs, **query_attrs):
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   658
        """yield user attributes for cwusers matching the given query_attrs
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   659
        (the result set cannot survive this method call)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   660
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   661
        This can be used by low-privileges account (anonymous comes to
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   662
        mind).
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   663
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   664
        `fetch_attrs`: tuple of attributes to be fetched
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   665
        `query_attrs`: dict of attr/values to restrict the query
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   666
        """
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   667
        assert query_attrs
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   668
        if not hasattr(self, '_cwuser_attrs'):
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   669
            cwuser = self.schema['CWUser']
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   670
            self._cwuser_attrs = set(str(rschema)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   671
                                     for rschema, _eschema in cwuser.attribute_definitions()
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   672
                                     if not rschema.meta)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   673
        cwuserattrs = self._cwuser_attrs
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   674
        for k in chain(fetch_attrs, query_attrs.iterkeys()):
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   675
            if k not in cwuserattrs:
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   676
                raise Exception('bad input for find_user')
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   677
        session = self.internal_session()
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   678
        try:
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   679
            varmaker = rqlvar_maker()
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   680
            vars = [(attr, varmaker.next()) for attr in fetch_attrs]
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   681
            rql = 'Any %s WHERE X is CWUser, ' % ','.join(var[1] for var in vars)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   682
            rql += ','.join('X %s %s' % (var[0], var[1]) for var in vars) + ','
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   683
            rset = session.execute(rql + ','.join('X %s %%(%s)s' % (attr, attr)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   684
                                                  for attr in query_attrs.iterkeys()),
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   685
                                   query_attrs)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   686
            return rset.rows
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   687
        finally:
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   688
            session.close()
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   689
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3629
diff changeset
   690
    def connect(self, login, **kwargs):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   691
        """open a connection for a given user
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   692
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   693
        base_url may be needed to send mails
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   694
        cnxtype indicate if this is a pyro connection or a in-memory connection
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   695
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   696
        raise `AuthenticationError` if the authentication failed
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   697
        raise `ConnectionError` if we can't open a connection
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   698
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   699
        # use an internal connection
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   700
        session = self.internal_session()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   701
        # try to get a user object
4089
ff92c7d692bf typos, api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3890
diff changeset
   702
        cnxprops = kwargs.pop('cnxprops', None)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   703
        try:
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3629
diff changeset
   704
            user = self.authenticate_user(session, login, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   705
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   706
            session.close()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   707
        session = Session(user, self, cnxprops)
3379
9192ba07890d use .cw_rset instead of rset on appobject classes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3293
diff changeset
   708
        user._cw = user.cw_rset.req = session
5557
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   709
        user.cw_clear_relation_cache()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   710
        self._sessions[session.id] = session
5638
cea831403e72 log the user name on session opening for easier debugging
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5603
diff changeset
   711
        self.info('opened session %s for user %s', session.id, login)
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
   712
        self.hm.call_hooks('session_open', session)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   713
        # commit session at this point in case write operation has been done
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   714
        # during `session_open` hooks
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   715
        session.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   716
        return session.id
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   717
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   718
    def execute(self, sessionid, rqlstring, args=None, build_descr=True,
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   719
                txid=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   720
        """execute a RQL query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   721
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   722
        * rqlstring should be an unicode string or a plain ascii string
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   723
        * args the optional parameters used in the query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   724
        * build_descr is a flag indicating if the description should be
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   725
          built on select queries
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   726
        """
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   727
        session = self._get_session(sessionid, setpool=True, txid=txid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   728
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   729
            try:
5747
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   730
                rset = self.querier.execute(session, rqlstring, args,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   731
                                            build_descr)
5747
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   732
                # NOTE: the web front will (re)build it when needed
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   733
                #       e.g in facets
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   734
                #       Zeroed to avoid useless overhead with pyro
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   735
                rset._rqlst = None
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   736
                return rset
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   737
            except (Unauthorized, RQLSyntaxError):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   738
                raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   739
            except ValidationError, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   740
                # need ValidationError normalization here so error may pass
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   741
                # through pyro
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   742
                if hasattr(ex.entity, 'eid'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   743
                    ex.entity = ex.entity.eid # error raised by yams
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   744
                    args = list(ex.args)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   745
                    args[0] = ex.entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   746
                    ex.args = tuple(args)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   747
                raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   748
            except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   749
                # FIXME: check error to catch internal errors
5167
529861a73ec8 log RQL query in case of unexpected failure (priceless when porting to new database)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5115
diff changeset
   750
                self.exception('unexpected error while executing %s with %s', rqlstring, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   751
                raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   752
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   753
            session.reset_pool()
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   754
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   755
    def describe(self, sessionid, eid, txid=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   756
        """return a tuple (type, source, extid) for the entity with id <eid>"""
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   757
        session = self._get_session(sessionid, setpool=True, txid=txid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   758
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   759
            return self.type_and_source_from_eid(eid, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   760
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   761
            session.reset_pool()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   762
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   763
    def check_session(self, sessionid):
6012
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   764
        """raise `BadConnectionId` if the connection is no more valid, else
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   765
        return its latest activity timestamp.
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   766
        """
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   767
        return self._get_session(sessionid, setpool=False).timestamp
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   768
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   769
    def get_shared_data(self, sessionid, key, default=None, pop=False, txdata=False):
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   770
        """return value associated to key in the session's data dictionary or
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   771
        session's transaction's data if `txdata` is true.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   772
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   773
        If pop is True, value will be removed from the dictionnary.
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   774
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   775
        If key isn't defined in the dictionnary, value specified by the
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   776
        `default` argument will be returned.
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   777
        """
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   778
        session = self._get_session(sessionid, setpool=False)
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   779
        return session.get_shared_data(key, default, pop, txdata)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   780
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   781
    def set_shared_data(self, sessionid, key, value, txdata=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   782
        """set value associated to `key` in shared data
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   783
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   784
        if `txdata` is true, the value will be added to the repository session's
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   785
        transaction's data which are cleared on commit/rollback of the current
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   786
        transaction.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   787
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   788
        session = self._get_session(sessionid, setpool=False)
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   789
        session.set_shared_data(key, value, txdata)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   790
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   791
    def commit(self, sessionid, txid=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   792
        """commit transaction for the session with the given id"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   793
        self.debug('begin commit for session %s', sessionid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   794
        try:
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   795
            session = self._get_session(sessionid)
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   796
            session.set_tx_data(txid)
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   797
            return session.commit()
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   798
        except (ValidationError, Unauthorized):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   799
            raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   800
        except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   801
            self.exception('unexpected error')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   802
            raise
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   803
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   804
    def rollback(self, sessionid, txid=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   805
        """commit transaction for the session with the given id"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   806
        self.debug('begin rollback for session %s', sessionid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   807
        try:
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   808
            session = self._get_session(sessionid)
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   809
            session.set_tx_data(txid)
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   810
            session.rollback()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   811
        except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   812
            self.exception('unexpected error')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   813
            raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   814
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   815
    def close(self, sessionid, txid=None, checkshuttingdown=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   816
        """close the session with the given id"""
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   817
        session = self._get_session(sessionid, setpool=True, txid=txid,
1939
67e7379edd96 #343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1883
diff changeset
   818
                                    checkshuttingdown=checkshuttingdown)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   819
        # operation uncommited before close are rollbacked before hook is called
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   820
        session.rollback(reset_pool=False)
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
   821
        self.hm.call_hooks('session_close', session)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   822
        # commit session at this point in case write operation has been done
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   823
        # during `session_close` hooks
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   824
        session.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   825
        session.close()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   826
        del self._sessions[sessionid]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   827
        self.info('closed session %s for user %s', sessionid, session.user.login)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   828
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   829
    def user_info(self, sessionid, props=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   830
        """this method should be used by client to:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   831
        * check session id validity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   832
        * update user information on each user's request (i.e. groups and
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   833
          custom properties)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   834
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   835
        session = self._get_session(sessionid, setpool=False)
2245
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   836
        if props is not None:
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   837
            self.set_session_props(sessionid, props)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   838
        user = session.user
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   839
        return user.eid, user.login, user.groups, user.properties
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   840
2245
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   841
    def set_session_props(self, sessionid, props):
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   842
        """this method should be used by client to:
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   843
        * check session id validity
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   844
        * update user information on each user's request (i.e. groups and
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   845
          custom properties)
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   846
        """
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   847
        session = self._get_session(sessionid, setpool=False)
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   848
        for prop, value in props.items():
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   849
            session.change_property(prop, value)
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   850
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   851
    def undoable_transactions(self, sessionid, ueid=None, txid=None,
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   852
                              **actionfilters):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   853
        """See :class:`cubicweb.dbapi.Connection.undoable_transactions`"""
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   854
        session = self._get_session(sessionid, setpool=True, txid=txid)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   855
        try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   856
            return self.system_source.undoable_transactions(session, ueid,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   857
                                                            **actionfilters)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   858
        finally:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   859
            session.reset_pool()
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   860
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   861
    def transaction_info(self, sessionid, txuuid, txid=None):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   862
        """See :class:`cubicweb.dbapi.Connection.transaction_info`"""
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   863
        session = self._get_session(sessionid, setpool=True, txid=txid)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   864
        try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   865
            return self.system_source.tx_info(session, txuuid)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   866
        finally:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   867
            session.reset_pool()
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   868
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   869
    def transaction_actions(self, sessionid, txuuid, public=True, txid=None):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   870
        """See :class:`cubicweb.dbapi.Connection.transaction_actions`"""
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   871
        session = self._get_session(sessionid, setpool=True, txid=txid)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   872
        try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   873
            return self.system_source.tx_actions(session, txuuid, public)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   874
        finally:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   875
            session.reset_pool()
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   876
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   877
    def undo_transaction(self, sessionid, txuuid, txid=None):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   878
        """See :class:`cubicweb.dbapi.Connection.undo_transaction`"""
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   879
        session = self._get_session(sessionid, setpool=True, txid=txid)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   880
        try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   881
            return self.system_source.undo_transaction(session, txuuid)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   882
        finally:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   883
            session.reset_pool()
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   884
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   885
    # public (inter-repository) interface #####################################
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   886
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   887
    def entities_modified_since(self, etypes, mtime):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   888
        """function designed to be called from an external repository which
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   889
        is using this one as a rql source for synchronization, and return a
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   890
        3-uple containing :
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   891
        * the local date
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   892
        * list of (etype, eid) of entities of the given types which have been
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   893
          modified since the given timestamp (actually entities whose full text
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   894
          index content has changed)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   895
        * list of (etype, eid) of entities of the given types which have been
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   896
          deleted since the given timestamp
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   897
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   898
        session = self.internal_session()
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 636
diff changeset
   899
        updatetime = datetime.now()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   900
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   901
            modentities, delentities = self.system_source.modified_entities(
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   902
                session, etypes, mtime)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   903
            return updatetime, modentities, delentities
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   904
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   905
            session.close()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   906
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   907
    # session handling ########################################################
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   908
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   909
    def close_sessions(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   910
        """close every opened sessions"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   911
        for sessionid in self._sessions.keys():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   912
            try:
1939
67e7379edd96 #343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1883
diff changeset
   913
                self.close(sessionid, checkshuttingdown=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   914
            except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   915
                self.exception('error while closing session %s' % sessionid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   916
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   917
    def clean_sessions(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   918
        """close sessions not used since an amount of time specified in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   919
        configuration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   920
        """
6012
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   921
        mintime = time() - self.cleanup_session_time
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   922
        self.debug('cleaning session unused since %s',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   923
                   strftime('%T', localtime(mintime)))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   924
        nbclosed = 0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   925
        for session in self._sessions.values():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   926
            if session.timestamp < mintime:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   927
                self.close(session.id)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   928
                nbclosed += 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   929
        return nbclosed
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   930
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   931
    def internal_session(self, cnxprops=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   932
        """return a dbapi like connection/cursor using internal user which
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   933
        have every rights on the repository. You'll *have to* commit/rollback
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   934
        or close (rollback implicitly) the session once the job's done, else
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   935
        you'll leak connections pool up to the time where no more pool is
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   936
        available, causing irremediable freeze...
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   937
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   938
        session = InternalSession(self, cnxprops)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   939
        session.set_pool()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   940
        return session
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   941
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   942
    def _get_session(self, sessionid, setpool=False, txid=None,
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   943
                     checkshuttingdown=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   944
        """return the user associated to the given session identifier"""
5906
d40ced753291 [repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5813
diff changeset
   945
        if checkshuttingdown and self.shutting_down:
1883
011e13d74cfc shuting -> shutting
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 1880
diff changeset
   946
            raise Exception('Repository is shutting down')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   947
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   948
            session = self._sessions[sessionid]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   949
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   950
            raise BadConnectionId('No such session %s' % sessionid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   951
        if setpool:
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   952
            session.set_tx_data(txid) # must be done before set_pool
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   953
            session.set_pool()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   954
        return session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   955
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   956
    # data sources handling ###################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   957
    # * correspondance between eid and (type, source)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   958
    # * correspondance between eid and local id (i.e. specific to a given source)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   959
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   960
    def type_and_source_from_eid(self, eid, session=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   961
        """return a tuple (type, source, extid) for the entity with id <eid>"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   962
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   963
            eid = typed_eid(eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   964
        except ValueError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   965
            raise UnknownEid(eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   966
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   967
            return self._type_source_cache[eid]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   968
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   969
            if session is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   970
                session = self.internal_session()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   971
                reset_pool = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   972
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   973
                reset_pool = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   974
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   975
                etype, uri, extid = self.system_source.eid_type_source(session,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   976
                                                                       eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   977
            finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   978
                if reset_pool:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   979
                    session.reset_pool()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   980
        self._type_source_cache[eid] = (etype, uri, extid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   981
        if uri != 'system':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   982
            self._extid_cache[(extid, uri)] = eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   983
        return etype, uri, extid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   984
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   985
    def clear_caches(self, eids):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   986
        etcache = self._type_source_cache
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   987
        extidcache = self._extid_cache
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   988
        rqlcache = self.querier._rql_cache
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   989
        for eid in eids:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   990
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   991
                etype, uri, extid = etcache.pop(typed_eid(eid)) # may be a string in some cases
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   992
                rqlcache.pop('%s X WHERE X eid %s' % (etype, eid), None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   993
                extidcache.pop((extid, uri), None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   994
            except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   995
                etype = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   996
            rqlcache.pop('Any X WHERE X eid %s' % eid, None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   997
            for source in self.sources:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   998
                source.clear_eid_cache(eid, etype)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   999
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1000
    def type_from_eid(self, eid, session=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1001
        """return the type of the entity with id <eid>"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1002
        return self.type_and_source_from_eid(eid, session)[0]
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1003
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1004
    def source_from_eid(self, eid, session=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1005
        """return the source for the given entity's eid"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1006
        return self.sources_by_uri[self.type_and_source_from_eid(eid, session)[1]]
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1007
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1008
    def querier_cache_key(self, session, rql, args, eidkeys):
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1009
        cachekey = [rql]
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1010
        for key in sorted(eidkeys):
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1011
            try:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1012
                etype = self.type_from_eid(args[key], session)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1013
            except KeyError:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1014
                raise QueryError('bad cache key %s (no value)' % key)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1015
            except TypeError:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1016
                raise QueryError('bad cache key %s (value: %r)' % (
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1017
                    key, args[key]))
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1018
            cachekey.append(etype)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1019
            # ensure eid is correctly typed in args
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1020
            args[key] = typed_eid(args[key])
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1021
        return tuple(cachekey)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1022
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1023
    def eid2extid(self, source, eid, session=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1024
        """get local id from an eid"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1025
        etype, uri, extid = self.type_and_source_from_eid(eid, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1026
        if source.uri != uri:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1027
            # eid not from the given source
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1028
            raise UnknownEid(eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1029
        return extid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1030
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1031
    def extid2eid(self, source, extid, etype, session=None, insert=True,
1250
5c20a7f13c84 new recreate argument to extid2eid when an external source want to recreate entities previously imported with a predictable ext id
sylvain.thenault@logilab.fr
parents: 1228
diff changeset
  1032
                  recreate=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1033
        """get eid from a local id. An eid is attributed if no record is found"""
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1034
        cachekey = (extid, source.uri)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1035
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1036
            return self._extid_cache[cachekey]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1037
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1038
            pass
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1039
        reset_pool = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1040
        if session is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1041
            session = self.internal_session()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1042
            reset_pool = True
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1043
        eid = self.system_source.extid2eid(session, source, extid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1044
        if eid is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1045
            self._extid_cache[cachekey] = eid
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1046
            self._type_source_cache[eid] = (etype, source.uri, extid)
5398
b9e1abe1bdfe [repo] cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5385
diff changeset
  1047
            # XXX used with extlite (eg vcsfile), probably not needed anymore
1250
5c20a7f13c84 new recreate argument to extid2eid when an external source want to recreate entities previously imported with a predictable ext id
sylvain.thenault@logilab.fr
parents: 1228
diff changeset
  1048
            if recreate:
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1049
                entity = source.before_entity_insertion(session, extid, etype, eid)
1250
5c20a7f13c84 new recreate argument to extid2eid when an external source want to recreate entities previously imported with a predictable ext id
sylvain.thenault@logilab.fr
parents: 1228
diff changeset
  1050
                entity._cw_recreating = True
5c20a7f13c84 new recreate argument to extid2eid when an external source want to recreate entities previously imported with a predictable ext id
sylvain.thenault@logilab.fr
parents: 1228
diff changeset
  1051
                if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1052
                    self.hm.call_hooks('before_add_entity', session, entity=entity)
1250
5c20a7f13c84 new recreate argument to extid2eid when an external source want to recreate entities previously imported with a predictable ext id
sylvain.thenault@logilab.fr
parents: 1228
diff changeset
  1053
                # XXX add fti op ?
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1054
                source.after_entity_insertion(session, extid, entity)
1250
5c20a7f13c84 new recreate argument to extid2eid when an external source want to recreate entities previously imported with a predictable ext id
sylvain.thenault@logilab.fr
parents: 1228
diff changeset
  1055
                if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1056
                    self.hm.call_hooks('after_add_entity', session, entity=entity)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1057
            if reset_pool:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1058
                session.reset_pool()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1059
            return eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1060
        if not insert:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1061
            return
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1062
        # no link between extid and eid, create one using an internal session
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1063
        # since the current session user may not have required permissions to
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1064
        # do necessary stuff and we don't want to commit user session.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1065
        #
450
5e14ea0e81c8 a note for later
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 341
diff changeset
  1066
        # Moreover, even if session is already an internal session but is
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1067
        # processing a commit, we have to use another one
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1068
        if not session.is_internal_session:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1069
            session = self.internal_session()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1070
            reset_pool = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1071
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1072
            eid = self.system_source.create_eid(session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1073
            self._extid_cache[cachekey] = eid
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1074
            self._type_source_cache[eid] = (etype, source.uri, extid)
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1075
            entity = source.before_entity_insertion(session, extid, etype, eid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1076
            if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1077
                self.hm.call_hooks('before_add_entity', session, entity=entity)
450
5e14ea0e81c8 a note for later
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 341
diff changeset
  1078
            # XXX call add_info with complete=False ?
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1079
            self.add_info(session, entity, source, extid)
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1080
            source.after_entity_insertion(session, extid, entity)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1081
            if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1082
                self.hm.call_hooks('after_add_entity', session, entity=entity)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1083
            session.commit(reset_pool)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1084
            return eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1085
        except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1086
            session.rollback(reset_pool)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1087
            raise
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1088
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1089
    def add_info(self, session, entity, source, extid=None, complete=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1090
        """add type and source info for an eid into the system table,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1091
        and index the entity with the full text index
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1092
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1093
        # begin by inserting eid/type/source/extid into the entities table
6426
541659c39f6a [hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6390
diff changeset
  1094
        hook.CleanupNewEidsCacheOp.get_instance(session).add_data(entity.eid)
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4756
diff changeset
  1095
        self.system_source.add_info(session, entity, source, extid, complete)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1096
6626
5c20a005bddc [pyro source] when cleaning relations of an external entity being cleaned up, only delete local relationsd
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6491
diff changeset
  1097
    def delete_info(self, session, entity, sourceuri, extid, scleanup=False):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1098
        """called by external source when some entity known by the system source
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1099
        has been deleted in the external source
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1100
        """
5067
adc2122eed03 [repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5043
diff changeset
  1101
        # mark eid as being deleted in session info and setup cache update
adc2122eed03 [repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5043
diff changeset
  1102
        # operation
6426
541659c39f6a [hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6390
diff changeset
  1103
        hook.CleanupDeletedEidsCacheOp.get_instance(session).add_data(entity.eid)
6626
5c20a005bddc [pyro source] when cleaning relations of an external entity being cleaned up, only delete local relationsd
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6491
diff changeset
  1104
        self._delete_info(session, entity, sourceuri, extid, scleanup)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1105
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1106
    def delete_info_multi(self, session, entities, sourceuri, extids, scleanup=False):
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1107
        """same as delete_info but accepts a list of entities and
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1108
        extids with the same etype and belonging to the same source
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1109
        """
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1110
        # mark eid as being deleted in session info and setup cache update
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1111
        # operation
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1112
        op = hook.CleanupDeletedEidsCacheOp.get_instance(session)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1113
        for entity in entities:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1114
            op.add_data(entity.eid)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1115
        self._delete_info_multi(session, entities, sourceuri, extids, scleanup)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1116
6629
edaa96d031b2 [repo] fix crash introduced in 6625:5c20a005bddc
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6626
diff changeset
  1117
    def _delete_info(self, session, entity, sourceuri, extid, scleanup=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1118
        """delete system information on deletion of an entity:
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1119
        * delete all remaining relations from/to this entity
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1120
        * call delete info on the system source which will transfer record from
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1121
          the entities table to the deleted_entities table
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1122
        """
2641
9c33d98a074e R [schema hooks] big refactoring / reorganization for clearer code, a few fixes on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2631
diff changeset
  1123
        pendingrtypes = session.transaction_data.get('pendingrtypes', ())
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1124
        # delete remaining relations: if user can delete the entity, he can
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1125
        # delete all its relations without security checking
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1126
        with security_enabled(session, read=False, write=False):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1127
            eid = entity.eid
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1128
            for rschema, _, role in entity.e_schema.relation_definitions():
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1129
                rtype = rschema.type
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1130
                if rtype in schema.VIRTUAL_RTYPES or rtype in pendingrtypes:
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1131
                    continue
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1132
                if role == 'subject':
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1133
                    # don't skip inlined relation so they are regularly
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1134
                    # deleted and so hooks are correctly called
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
  1135
                    rql = 'DELETE X %s Y WHERE X eid %%(x)s' % rtype
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1136
                else:
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
  1137
                    rql = 'DELETE Y %s X WHERE X eid %%(x)s' % rtype
6626
5c20a005bddc [pyro source] when cleaning relations of an external entity being cleaned up, only delete local relationsd
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6491
diff changeset
  1138
                if scleanup:
5c20a005bddc [pyro source] when cleaning relations of an external entity being cleaned up, only delete local relationsd
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6491
diff changeset
  1139
                    # source cleaning: only delete relations stored locally
6674
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1140
                    rql += ', NOT (Y cw_source S, S name %(source)s)'
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1141
                try:
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1142
                    session.execute(rql, {'x': eid, 'source': sourceuri},
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1143
                                    build_descr=False)
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1144
                except:
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1145
                    self.exception('error while cascading delete for entity %s '
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1146
                                   'from %s. RQL: %s', entity, sourceuri, rql)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1147
        self.system_source.delete_info(session, entity, sourceuri, extid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1148
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1149
    def _delete_info_multi(self, session, entities, sourceuri, extids, scleanup=False):
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1150
        """same as _delete_info but accepts a list of entities with
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1151
        the same etype and belinging to the same source.
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1152
        """
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1153
        pendingrtypes = session.transaction_data.get('pendingrtypes', ())
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1154
        # delete remaining relations: if user can delete the entity, he can
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1155
        # delete all its relations without security checking
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1156
        assert entities and len(entities) == len(extids)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1157
        with security_enabled(session, read=False, write=False):
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1158
            eids = [_e.eid for _e in entities]
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1159
            in_eids = ','.join((str(eid) for eid in eids))
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1160
            for rschema, _, role in entities[0].e_schema.relation_definitions():
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1161
                rtype = rschema.type
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1162
                if rtype in schema.VIRTUAL_RTYPES or rtype in pendingrtypes:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1163
                    continue
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1164
                if role == 'subject':
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1165
                    # don't skip inlined relation so they are regularly
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1166
                    # deleted and so hooks are correctly called
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1167
                    rql = 'DELETE X %s Y WHERE X eid IN (%s)' % (rtype, in_eids)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1168
                else:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1169
                    rql = 'DELETE Y %s X WHERE X eid IN (%s)' % (rtype, in_eids)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1170
                if scleanup:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1171
                    # source cleaning: only delete relations stored locally
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1172
                    rql += ', NOT (Y cw_source S, S name %(source)s)'
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1173
                try:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1174
                    session.execute(rql, {'source': sourceuri},
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1175
                                    build_descr=False)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1176
                except:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1177
                    self.exception('error while cascading delete for entity %s '
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1178
                                   'from %s. RQL: %s', entities, sourceuri, rql)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1179
        self.system_source.delete_info_multi(session, entities, sourceuri, extids)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1180
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1181
    def locate_relation_source(self, session, subject, rtype, object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1182
        subjsource = self.source_from_eid(subject, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1183
        objsource = self.source_from_eid(object, session)
3042
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1184
        if not subjsource is objsource:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1185
            source = self.system_source
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1186
            if not (subjsource.may_cross_relation(rtype)
3042
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1187
                    and objsource.may_cross_relation(rtype)):
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1188
                raise MultiSourcesError(
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1189
                    "relation %s can't be crossed among sources"
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1190
                    % rtype)
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1191
        elif not subjsource.support_relation(rtype):
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1192
            source = self.system_source
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1193
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1194
            source = subjsource
3042
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1195
        if not source.support_relation(rtype, True):
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1196
            raise MultiSourcesError(
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1197
                "source %s doesn't support write of %s relation"
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1198
                % (source.uri, rtype))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1199
        return source
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1200
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1201
    def locate_etype_source(self, etype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1202
        for source in self.sources:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1203
            if source.support_entity(etype, 1):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1204
                return source
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1205
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1206
            raise ETypeNotSupportedBySources(etype)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1207
5068
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1208
    def init_entity_caches(self, session, entity, source):
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1209
        """add entity to session entities cache and repo's extid cache.
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1210
        Return entity's ext id if the source isn't the system source.
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1211
        """
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1212
        session.set_entity_cache(entity)
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1213
        suri = source.uri
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1214
        if suri == 'system':
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1215
            extid = None
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1216
        else:
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1217
            extid = source.get_extid(entity)
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1218
            self._extid_cache[(str(extid), suri)] = entity.eid
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1219
        self._type_source_cache[entity.eid] = (entity.__regid__, suri, extid)
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1220
        return extid
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1221
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1222
    def glob_add_entity(self, session, edited):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1223
        """add an entity to the repository
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1224
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1225
        the entity eid should originaly be None and a unique eid is assigned to
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1226
        the entity instance
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1227
        """
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1228
        entity = edited.entity
5557
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1229
        entity._cw_is_saved = False # entity has an eid but is not yet saved
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1230
        # init edited_attributes before calling before_add_entity hooks
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1231
        entity.cw_edited = edited
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1232
        eschema = entity.e_schema
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
  1233
        source = self.locate_etype_source(entity.__regid__)
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
  1234
        # allocate an eid to the entity before calling hooks
5557
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1235
        entity.eid = self.system_source.create_eid(session)
5068
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1236
        # set caches asap
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1237
        extid = self.init_entity_caches(session, entity, source)
2600
6cd6c5d11b45 [F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
  1238
        if server.DEBUG & server.DBG_REPO:
6466
43d71dd8a8ec cleanup and 3.10 updates
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6465
diff changeset
  1239
            print 'ADD entity', self, entity.__regid__, entity.eid, edited
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1240
        relations = []
2929
51cdfe069e10 fix edited_attributes handling when adding entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2875
diff changeset
  1241
        if source.should_call_hooks:
2968
0e3460341023 somewhat painful backport of 3.5 branch, should mostly be ok
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2893 2967
diff changeset
  1242
            self.hm.call_hooks('before_add_entity', session, entity=entity)
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1243
        for attr in edited.iterkeys():
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3606
diff changeset
  1244
            rschema = eschema.subjrels[attr]
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3606
diff changeset
  1245
            if not rschema.final: # inlined relation
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1246
                relations.append((attr, edited[attr]))
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1247
        edited.set_defaults()
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
  1248
        if session.is_hook_category_activated('integrity'):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1249
            edited.check(creation=True)
6464
11f9fbf6a645 [repo, entity] move entity cache initialization to a function, and call it before source.add_entity so it may be used in error handler or such
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
  1250
        prefill_entity_caches(entity, relations)
6211
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1251
        try:
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1252
            source.add_entity(session, entity)
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1253
        except UniqueTogetherError, exc:
6465
6401a9d0b5aa [architecture] introduce adapter to ease transformation of errors before display to the end user. Use it for UniqueTogetherError first
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6464
diff changeset
  1254
            userhdlr = session.vreg['adapters'].select(
6401a9d0b5aa [architecture] introduce adapter to ease transformation of errors before display to the end user. Use it for UniqueTogetherError first
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6464
diff changeset
  1255
                'IUserFriendlyError', session, entity=entity, exc=exc)
6401a9d0b5aa [architecture] introduce adapter to ease transformation of errors before display to the end user. Use it for UniqueTogetherError first
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6464
diff changeset
  1256
            userhdlr.raise_user_exception()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1257
        self.add_info(session, entity, source, extid, complete=False)
6368
f907cc7f2875 [repo] properly mark object as saved once added to its source (code much probably gone during a merge...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6333
diff changeset
  1258
        edited.saved = entity._cw_is_saved = True
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1259
        # trigger after_add_entity after after_add_relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1260
        if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1261
            self.hm.call_hooks('after_add_entity', session, entity=entity)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1262
            # call hooks for inlined relations
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1263
            for attr, value in relations:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1264
                self.hm.call_hooks('before_add_relation', session,
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1265
                                    eidfrom=entity.eid, rtype=attr, eidto=value)
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1266
                self.hm.call_hooks('after_add_relation', session,
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1267
                                    eidfrom=entity.eid, rtype=attr, eidto=value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1268
        return entity.eid
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1269
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1270
    def glob_update_entity(self, session, edited):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1271
        """replace an entity in the repository
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1272
        the type and the eid of an entity must not be changed
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1273
        """
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1274
        entity = edited.entity
2600
6cd6c5d11b45 [F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
  1275
        if server.DEBUG & server.DBG_REPO:
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
  1276
            print 'UPDATE entity', entity.__regid__, entity.eid, \
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1277
                  entity.cw_attr_cache, edited
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1278
        hm = self.hm
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1279
        eschema = entity.e_schema
2647
b0a2e779845c enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2641
diff changeset
  1280
        session.set_entity_cache(entity)
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1281
        orig_edited = getattr(entity, 'cw_edited', None)
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1282
        entity.cw_edited = edited
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1283
        try:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1284
            only_inline_rels, need_fti_update = True, False
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1285
            relations = []
5292
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1286
            source = self.source_from_eid(entity.eid, session)
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1287
            for attr in list(edited):
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1288
                if attr == 'eid':
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1289
                    continue
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1290
                rschema = eschema.subjrels[attr]
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1291
                if rschema.final:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1292
                    if getattr(eschema.rdef(attr), 'fulltextindexed', False):
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1293
                        need_fti_update = True
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1294
                    only_inline_rels = False
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1295
                else:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1296
                    # inlined relation
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1297
                    previous_value = entity.related(attr) or None
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1298
                    if previous_value is not None:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1299
                        previous_value = previous_value[0][0] # got a result set
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1300
                        if previous_value == entity.cw_attr_cache[attr]:
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1301
                            previous_value = None
5292
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1302
                        elif source.should_call_hooks:
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1303
                            hm.call_hooks('before_delete_relation', session,
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1304
                                          eidfrom=entity.eid, rtype=attr,
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1305
                                          eidto=previous_value)
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1306
                    relations.append((attr, edited[attr], previous_value))
5292
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1307
            if source.should_call_hooks:
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1308
                # call hooks for inlined relations
6211
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1309
                for attr, value, _t in relations:
5292
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1310
                    hm.call_hooks('before_add_relation', session,
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1311
                                  eidfrom=entity.eid, rtype=attr, eidto=value)
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1312
                if not only_inline_rels:
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1313
                    hm.call_hooks('before_update_entity', session, entity=entity)
5808
2de32c0c293b [repo] call entity.check() once before_update hooks has been called, to let them a chance to transform data
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5749
diff changeset
  1314
            if session.is_hook_category_activated('integrity'):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1315
                edited.check()
6211
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1316
            try:
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1317
                source.update_entity(session, entity)
6225
a176e68b7d0d backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6142 6211
diff changeset
  1318
                edited.saved = True
6211
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1319
            except UniqueTogetherError, exc:
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1320
                etype, rtypes = exc.args
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1321
                problems = {}
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1322
                for col in rtypes:
6225
a176e68b7d0d backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6142 6211
diff changeset
  1323
                    problems[col] = session._('violates unique_together constraints (%s)') % (','.join(rtypes))
6211
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1324
                raise ValidationError(entity.eid, problems)
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1325
            self.system_source.update_info(session, entity, need_fti_update)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1326
            if source.should_call_hooks:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1327
                if not only_inline_rels:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1328
                    hm.call_hooks('after_update_entity', session, entity=entity)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1329
                for attr, value, prevvalue in relations:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1330
                    # if the relation is already cached, update existant cache
5557
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1331
                    relcache = entity.cw_relation_cached(attr, 'subject')
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1332
                    if prevvalue is not None:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1333
                        hm.call_hooks('after_delete_relation', session,
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1334
                                      eidfrom=entity.eid, rtype=attr, eidto=prevvalue)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1335
                        if relcache is not None:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1336
                            session.update_rel_cache_del(entity.eid, attr, prevvalue)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1337
                    del_existing_rel_if_needed(session, entity.eid, attr, value)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1338
                    if relcache is not None:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1339
                        session.update_rel_cache_add(entity.eid, attr, value)
2647
b0a2e779845c enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2641
diff changeset
  1340
                    else:
5557
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1341
                        entity.cw_set_relation_cache(attr, 'subject',
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1342
                                                     session.eid_rset(value))
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1343
                    hm.call_hooks('after_add_relation', session,
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1344
                                  eidfrom=entity.eid, rtype=attr, eidto=value)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1345
        finally:
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1346
            if orig_edited is not None:
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1347
                entity.cw_edited = orig_edited
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1348
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1349
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1350
    def glob_delete_entities(self, session, eids):
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1351
        """delete a list of  entities and all related entities from the repository"""
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1352
        data_by_etype_source = {} # values are ([list of eids],
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1353
                                  #             [list of extid],
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1354
                                  #             [list of entities])
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1355
        #
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1356
        # WARNING: the way this dictionary is populated is heavily optimized
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1357
        # and does not use setdefault on purpose. Unless a new release
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1358
        # of the Python interpreter advertises large perf improvements
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1359
        # in setdefault, this should not be changed without profiling.
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1360
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1361
        for eid in eids:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1362
            etype, sourceuri, extid = self.type_and_source_from_eid(eid, session)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1363
            entity = session.entity_from_eid(eid, etype)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1364
            _key = (etype, sourceuri)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1365
            if _key not in data_by_etype_source:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1366
                data_by_etype_source[_key] = ([eid], [extid], [entity])
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1367
            else:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1368
                _data = data_by_etype_source[_key]
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1369
                _data[0].append(eid)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1370
                _data[1].append(extid)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1371
                _data[2].append(entity)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1372
        for (etype, sourceuri), (eids, extids, entities) in data_by_etype_source.iteritems():
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1373
            if server.DEBUG & server.DBG_REPO:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1374
                print 'DELETE entities', etype, eids
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1375
            #print 'DELETE entities', etype, len(eids)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1376
            source = self.sources_by_uri[sourceuri]
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1377
            if source.should_call_hooks:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1378
                self.hm.call_hooks('before_delete_entity', session, entities=entities)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1379
            self._delete_info_multi(session, entities, sourceuri, extids) # xxx
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1380
            source.delete_entities(session, entities)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1381
            if source.should_call_hooks:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1382
                self.hm.call_hooks('after_delete_entity', session, entities=entities)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1383
        # don't clear cache here this is done in a hook on commit
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1384
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1385
    def glob_add_relation(self, session, subject, rtype, object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1386
        """add a relation to the repository"""
2600
6cd6c5d11b45 [F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
  1387
        if server.DEBUG & server.DBG_REPO:
6cd6c5d11b45 [F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
  1388
            print 'ADD relation', subject, rtype, object
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1389
        source = self.locate_relation_source(session, subject, rtype, object)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1390
        if source.should_call_hooks:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1391
            del_existing_rel_if_needed(session, subject, rtype, object)
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1392
            self.hm.call_hooks('before_add_relation', session,
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1393
                               eidfrom=subject, rtype=rtype, eidto=object)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1394
        source.add_relation(session, subject, rtype, object)
2647
b0a2e779845c enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2641
diff changeset
  1395
        rschema = self.schema.rschema(rtype)
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
  1396
        session.update_rel_cache_add(subject, rtype, object, rschema.symmetric)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1397
        if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1398
            self.hm.call_hooks('after_add_relation', session,
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1399
                               eidfrom=subject, rtype=rtype, eidto=object)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1400
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1401
    def glob_delete_relation(self, session, subject, rtype, object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1402
        """delete a relation from the repository"""
2600
6cd6c5d11b45 [F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
  1403
        if server.DEBUG & server.DBG_REPO:
6cd6c5d11b45 [F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
  1404
            print 'DELETE relation', subject, rtype, object
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1405
        source = self.locate_relation_source(session, subject, rtype, object)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1406
        if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1407
            self.hm.call_hooks('before_delete_relation', session,
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1408
                               eidfrom=subject, rtype=rtype, eidto=object)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1409
        source.delete_relation(session, subject, rtype, object)
2647
b0a2e779845c enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2641
diff changeset
  1410
        rschema = self.schema.rschema(rtype)
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
  1411
        session.update_rel_cache_del(subject, rtype, object, rschema.symmetric)
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
  1412
        if rschema.symmetric:
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
  1413
            # on symmetric relation, we can't now in which sense it's
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1414
            # stored so try to delete both
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1415
            source.delete_relation(session, object, rtype, subject)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1416
        if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1417
            self.hm.call_hooks('after_delete_relation', session,
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1418
                               eidfrom=subject, rtype=rtype, eidto=object)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1419
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1420
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1421
    # pyro handling ###########################################################
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1422
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1423
    @property
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1424
    @cached
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1425
    def pyro_appid(self):
6061
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1426
        from logilab.common import pyro_ext as pyro
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1427
        config = self.config
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1428
        appid = '%s.%s' % pyro.ns_group_and_id(
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1429
            config['pyro-instance-id'] or config.appid,
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1430
            config['pyro-ns-group'])
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1431
        # ensure config['pyro-instance-id'] is a full qualified pyro name
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1432
        config['pyro-instance-id'] = appid
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1433
        return appid
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1434
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1435
    def pyro_register(self, host=''):
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1436
        """register the repository as a pyro object"""
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1437
        from logilab.common import pyro_ext as pyro
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1438
        daemon = pyro.register_object(self, self.pyro_appid,
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1439
                                      daemonhost=self.config['pyro-host'],
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1440
                                      nshost=self.config['pyro-ns-host'])
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1441
        self.info('repository registered as a pyro object %s', self.pyro_appid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1442
        self.pyro_registered = True
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1443
        # register a looping task to regularly ensure we're still registered
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1444
        # into the pyro name server
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1445
        self.looping_task(60*10, self._ensure_pyro_ns)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1446
        return daemon
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1447
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1448
    def _ensure_pyro_ns(self):
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1449
        from logilab.common import pyro_ext as pyro
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1450
        pyro.ns_reregister(self.pyro_appid, nshost=self.config['pyro-ns-host'])
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1451
        self.info('repository re-registered as a pyro object %s',
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1452
                  self.pyro_appid)
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1453
1228
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1454
    # multi-sources planner helpers ###########################################
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1455
1228
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1456
    @cached
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1457
    def rel_type_sources(self, rtype):
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1458
        return tuple([source for source in self.sources
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1459
                      if source.support_relation(rtype)
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1460
                      or rtype in source.dont_cross_relations])
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1461
1228
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1462
    @cached
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1463
    def can_cross_relation(self, rtype):
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1464
        return tuple([source for source in self.sources
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1465
                      if source.support_relation(rtype)
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1466
                      and rtype in source.cross_relations])
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1467
1228
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1468
    @cached
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1469
    def is_multi_sources_relation(self, rtype):
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1470
        return any(source for source in self.sources
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1471
                   if not source is self.system_source
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1472
                   and source.support_relation(rtype))
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1473
7083
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7064
diff changeset
  1474
    # these are overridden by set_log_methods below
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7064
diff changeset
  1475
    # only defining here to prevent pylint from complaining
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7064
diff changeset
  1476
    info = warning = error = critical = exception = debug = lambda msg,*a,**kw: None
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1477
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1478
def pyro_unregister(config):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1479
    """unregister the repository from the pyro name server"""
2665
0c6281487f90 [pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2650
diff changeset
  1480
    from logilab.common.pyro_ext import ns_unregister
3539
f3b14d052798 [pyro] merge pyro-id / pyro-instance-id options, put all pyro options in the same section of the configuration file
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3250
diff changeset
  1481
    appid = config['pyro-instance-id'] or config.appid
2665
0c6281487f90 [pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2650
diff changeset
  1482
    ns_unregister(appid, config['pyro-ns-group'], config['pyro-ns-host'])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1483
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1484
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1485
from logging import getLogger
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1486
from cubicweb import set_log_methods
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1487
set_log_methods(Repository, getLogger('cubicweb.repository'))