server/repository.py
author Alexandre Fayolle <alexandre.fayolle@logilab.fr>
Fri, 22 Apr 2011 10:29:23 +0200
changeset 7243 9ab01bf84eac
parent 7238 576abb8c4626
child 7305 1f97c05cfbab
permissions -rw-r--r--
fix bad handling of inlined relations when entities of different etypes are passed as subject to add_relations
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
     1
# copyright 2003-2011 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
"""Defines the central class for the CubicWeb RQL server: the repository.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
The repository is an abstraction allowing execution of rql queries against
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
data sources. Most of the work is actually done in helper classes. The
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
repository mainly:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
* brings these classes all together to provide a single access
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2473
diff changeset
    25
  point to a cubicweb instance.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
* handles session management
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
* provides method for pyro registration, to call if pyro is enabled
5587
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
    28
"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    29
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
    30
from __future__ import with_statement
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
    31
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    32
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    33
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    34
import sys
5587
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
    35
import threading
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    36
import Queue
6390
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
    37
from itertools import chain
4719
aaed3f813ef8 kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4714
diff changeset
    38
from os.path import join
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 636
diff changeset
    39
from datetime import datetime
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    40
from time import time, localtime, strftime
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    41
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
    42
from logilab.common.decorators import cached, clear_cache
3757
122a01751d59 2.4 compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3694
diff changeset
    43
from logilab.common.compat import any
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
    44
from logilab.common import flatten
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    45
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    46
from yams import BadSchemaDefinition
5030
5238d9a8dfee [form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4984
diff changeset
    47
from yams.schema import role_name
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    48
from rql import RQLSyntaxError
6390
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
    49
from rql.utils import rqlvar_maker
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    50
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
    51
from cubicweb import (CW_SOFTWARE_ROOT, CW_MIGRATION_MAP, QueryError,
2709
6ee816eb9f25 [hooksmanager] hooks should be reseted after vreg reload
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2708
diff changeset
    52
                      UnknownEid, AuthenticationError, ExecutionError,
3042
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
    53
                      ETypeNotSupportedBySources, MultiSourcesError,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    54
                      BadConnectionId, Unauthorized, ValidationError,
6211
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
    55
                      RepositoryError, UniqueTogetherError, typed_eid, onevent)
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
    56
from cubicweb import cwvreg, schema, server
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
    57
from cubicweb.server import utils, hook, pool, querier, sources
5079
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
    58
from cubicweb.server.session import Session, InternalSession, InternalManager, \
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
    59
     security_enabled
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
    60
from cubicweb.server.ssplanner import EditedEntity
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    61
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
    62
6467
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    63
def prefill_entity_caches(entity, relations):
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    64
    session = entity._cw
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    65
    # prefill entity relation caches
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    66
    for rschema in entity.e_schema.subject_relations():
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    67
        rtype = str(rschema)
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    68
        if rtype in schema.VIRTUAL_RTYPES:
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    69
            continue
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    70
        if rschema.final:
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    71
            entity.cw_attr_cache.setdefault(rtype, None)
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    72
        else:
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    73
            entity.cw_set_relation_cache(rtype, 'subject',
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    74
                                         session.empty_rset())
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    75
    for rschema in entity.e_schema.object_relations():
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    76
        rtype = str(rschema)
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    77
        if rtype in schema.VIRTUAL_RTYPES:
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    78
            continue
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    79
        entity.cw_set_relation_cache(rtype, 'object', session.empty_rset())
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    80
    # set inlined relation cache before call to after_add_entity
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    81
    for attr, value in relations:
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    82
        session.update_rel_cache_add(entity.eid, attr, value)
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    83
        del_existing_rel_if_needed(session, entity.eid, attr, value)
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    84
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    85
def del_existing_rel_if_needed(session, eidfrom, rtype, eidto):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
    """delete existing relation when adding a new one if card is 1 or ?
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    88
    have to be done once the new relation has been inserted to avoid having
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    89
    an entity without a relation for some time
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
    this kind of behaviour has to be done in the repository so we don't have
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
    hooks order hazardness
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
    """
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
    94
    # skip that for internal session or if integrity explicitly disabled
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
    95
    #
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
    96
    # XXX we should imo rely on the orm to first fetch existing entity if any
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
    97
    # then delete it.
4756
a2c73b272c9b do not check for existant entity when relation of single card is added but integrity hooks are deactivated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4719
diff changeset
    98
    if session.is_internal_session \
5090
8c39d2bf58fd [repo creation] removing existing entities of 'single' cardinality relatino should be considered as 'activeintegrity' hook. Also don't disable that category during repo creation to avoid pb such as two default workflows for one entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5084
diff changeset
    99
           or not session.is_hook_category_activated('activeintegrity'):
4687
082e66184f71 [integrity] skip auto-delete for internal sessions (but *not* for super session, see 4642:921737d2e3a8)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4686
diff changeset
   100
        return
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   101
    card = session.schema_rproperty(rtype, eidfrom, eidto, 'cardinality')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   102
    # one may be tented to check for neweids but this may cause more than one
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   103
    # relation even with '1?'  cardinality if thoses relations are added in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   104
    # same transaction where the entity is being created. This never occurs from
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   105
    # the web interface but may occurs during test or dbapi connection (though
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   106
    # not expected for this).  So: don't do it, we pretend to ensure repository
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   107
    # consistency.
4673
6f8b925a29f4 auto-delete relation with single cardinality using execute, not unsafe_execute, so permissions are correctly checked
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4643
diff changeset
   108
    #
6189
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   109
    # notes:
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   110
    # * inlined relations will be implicitly deleted for the subject entity
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   111
    # * we don't want read permissions to be applied but we want delete
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   112
    #   permission to be checked
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   113
    if card[0] in '1?' and not session.repo.schema.rschema(rtype).inlined:
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   114
        with security_enabled(session, read=False):
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   115
            session.execute('DELETE X %s Y WHERE X eid %%(x)s, '
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   116
                            'NOT Y eid %%(y)s' % rtype,
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
   117
                                {'x': eidfrom, 'y': eidto})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   118
    if card[1] in '1?':
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   119
        with security_enabled(session, read=False):
5711
ea2538a3c622 [repo] fix typo
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5641
diff changeset
   120
            session.execute('DELETE X %s Y WHERE Y eid %%(y)s, '
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   121
                            'NOT X eid %%(x)s' % rtype,
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
   122
                            {'x': eidfrom, 'y': eidto})
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   123
3694
33dbb1da1db9 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   124
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   125
class Repository(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   126
    """a repository provides access to a set of persistent storages for
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   127
    entities and relations
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   128
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   129
    XXX protect pyro access
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   130
    """
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   131
5442
3ed8afbbdf70 [webconfig] refactor/cleanup debug mode management on startup: simply use config.debugmode instead of debug argument everywhere...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   132
    def __init__(self, config, vreg=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   133
        self.config = config
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   134
        if vreg is None:
5442
3ed8afbbdf70 [webconfig] refactor/cleanup debug mode management on startup: simply use config.debugmode instead of debug argument everywhere...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   135
            vreg = cwvreg.CubicWebVRegistry(config)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   136
        self.vreg = vreg
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   137
        self.pyro_registered = False
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
   138
        self.pyro_uri = None
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   139
        self.info('starting repository from %s', self.config.apphome)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   140
        # dictionary of opened sessions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   141
        self._sessions = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   142
        # list of functions to be called at regular interval
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   143
        self._looping_tasks = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   144
        # list of running threads
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   145
        self._running_threads = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   146
        # initial schema, should be build or replaced latter
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   147
        self.schema = schema.CubicWebSchema(config.appid)
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3042
diff changeset
   148
        self.vreg.schema = self.schema # until actual schema is loaded...
5906
d40ced753291 [repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5813
diff changeset
   149
        # shutdown flag
d40ced753291 [repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5813
diff changeset
   150
        self.shutting_down = False
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   151
        # sources (additional sources info in the system database)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   152
        self.system_source = self.get_source('native', 'system',
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   153
                                             config.sources()['system'])
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   154
        self.sources = [self.system_source]
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   155
        self.sources_by_uri = {'system': self.system_source}
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   156
        # querier helper, need to be created after sources initialization
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   157
        self.querier = querier.QuerierHelper(self, self.schema)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   158
        # cache eid -> type / source
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   159
        self._type_source_cache = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
        # cache (extid, source uri) -> eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   161
        self._extid_cache = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   162
        # open some connections pools
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2929
diff changeset
   163
        if config.open_connections_pools:
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2929
diff changeset
   164
            self.open_connections_pools()
5079
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   165
        @onevent('after-registry-reload', self)
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   166
        def fix_user_classes(self):
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   167
            usercls = self.vreg['etypes'].etype_class('CWUser')
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   168
            for session in self._sessions.values():
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   169
                if not isinstance(session.user, InternalManager):
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   170
                    session.user.__class__ = usercls
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2929
diff changeset
   171
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2929
diff changeset
   172
    def open_connections_pools(self):
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2929
diff changeset
   173
        config = self.config
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   174
        self._available_pools = Queue.Queue()
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   175
        self._available_pools.put_nowait(pool.ConnectionsPool(self.sources))
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   176
        if config.quick_start:
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   177
            # quick start, usually only to get a minimal repository to get cubes
5084
d8f491cb046c [repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5079
diff changeset
   178
            # information (eg dump/restore/...)
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   179
            config._cubes = ()
5084
d8f491cb046c [repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5079
diff changeset
   180
            # only load hooks and entity classes in the registry
d8f491cb046c [repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5079
diff changeset
   181
            config.cube_appobject_path = set(('hooks', 'entities'))
d8f491cb046c [repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5079
diff changeset
   182
            config.cubicweb_appobject_path = set(('hooks', 'entities'))
d8f491cb046c [repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5079
diff changeset
   183
            self.set_schema(config.load_schema())
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   184
            config['connections-pool-size'] = 1
5084
d8f491cb046c [repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5079
diff changeset
   185
            # will be reinitialized later from cubes found in the database
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   186
            config._cubes = None
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   187
        elif config.creating:
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   188
            # repository creation
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   189
            config.bootstrap_cubes()
2968
0e3460341023 somewhat painful backport of 3.5 branch, should mostly be ok
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2893 2967
diff changeset
   190
            self.set_schema(config.load_schema(), resetvreg=False)
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   191
            # need to load the Any and CWUser entity types
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   192
            etdirectory = join(CW_SOFTWARE_ROOT, 'entities')
1317
sylvain.thenault@logilab.fr
parents: 1316
diff changeset
   193
            self.vreg.init_registration([etdirectory])
3409
c684606a95e0 [bootstrap] metadata hooks need to be registered at bootstrap time for db creation
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3381
diff changeset
   194
            for modname in ('__init__', 'authobjs', 'wfobjs'):
c684606a95e0 [bootstrap] metadata hooks need to be registered at bootstrap time for db creation
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents: 3381
diff changeset
   195
                self.vreg.load_file(join(etdirectory, '%s.py' % modname),
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   196
                                    'cubicweb.entities.%s' % modname)
5447
f29dafba250a [vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5442
diff changeset
   197
            hooksdirectory = join(CW_SOFTWARE_ROOT, 'hooks')
f29dafba250a [vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5442
diff changeset
   198
            self.vreg.load_file(join(hooksdirectory, 'metadata.py'),
f29dafba250a [vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5442
diff changeset
   199
                                'cubicweb.hooks.metadata')
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   200
        elif config.read_instance_schema:
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   201
            # normal start: load the instance schema from the database
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   202
            self.fill_schema()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   203
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   204
            # test start: use the file system schema (quicker)
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2473
diff changeset
   205
            self.warning("set fs instance'schema")
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   206
            config.bootstrap_cubes()
2961
6d833a440575 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2959
diff changeset
   207
            self.set_schema(config.load_schema())
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   208
        if not config.creating:
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   209
            self.init_sources_from_database()
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   210
            if 'CWProperty' in self.schema:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   211
                self.vreg.init_properties(self.properties())
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   212
        else:
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
   213
            # call init_creating so that for instance native source can
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
   214
            # configurate tsearch according to postgres version
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   215
            for source in self.sources:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   216
                source.init_creating()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   217
        # close initialization pool and reopen fresh ones for proper
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   218
        # initialization now that we know cubes
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   219
        self._get_pool().close(True)
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   220
        # list of available pools (we can't iterate on Queue instance)
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   221
        self.pools = []
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   222
        for i in xrange(config['connections-pool-size']):
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   223
            self.pools.append(pool.ConnectionsPool(self.sources))
2493
9806571ea790 major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2476
diff changeset
   224
            self._available_pools.put_nowait(self.pools[-1])
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   225
        if config.quick_start:
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   226
            config.init_cubes(self.get_cubes())
5093
8d073d2e089d [optimization] improve massive write performance by optimizing hooks selection
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5090
diff changeset
   227
        self.hm = hook.HooksManager(self.vreg)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   228
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   229
    # internals ###############################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   230
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   231
    def init_sources_from_database(self):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   232
        self.sources_by_eid = {}
6664
7b6acf314a7a [repo] don't attempt to read sources from database on quick start. Fix crash after restoring a pre 3.10 database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6629
diff changeset
   233
        if self.config.quick_start \
7b6acf314a7a [repo] don't attempt to read sources from database on quick start. Fix crash after restoring a pre 3.10 database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6629
diff changeset
   234
               or not 'CWSource' in self.schema: # # 3.10 migration
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   235
            return
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   236
        session = self.internal_session()
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   237
        try:
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   238
            # FIXME: sources should be ordered (add_entity priority)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   239
            for sourceent in session.execute(
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   240
                'Any S, SN, SA, SC WHERE S is_instance_of CWSource, '
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   241
                'S name SN, S type SA, S config SC').entities():
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   242
                if sourceent.name == 'system':
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   243
                    self.system_source.eid = sourceent.eid
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   244
                    self.sources_by_eid[sourceent.eid] = self.system_source
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   245
                    self.system_source.init(True, sourceent)
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   246
                    continue
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   247
                self.add_source(sourceent, add_to_pools=False)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   248
        finally:
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   249
            session.close()
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   250
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   251
    def _clear_planning_caches(self):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   252
        for cache in ('source_defs', 'is_multi_sources_relation',
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   253
                      'can_cross_relation', 'rel_type_sources'):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   254
            clear_cache(self, cache)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   255
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   256
    def add_source(self, sourceent, add_to_pools=True):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   257
        source = self.get_source(sourceent.type, sourceent.name,
6945
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6941
diff changeset
   258
                                 sourceent.host_config, sourceent.eid)
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   259
        self.sources_by_eid[sourceent.eid] = source
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   260
        self.sources_by_uri[sourceent.name] = source
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   261
        if self.config.source_enabled(source):
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   262
            # call source's init method to complete their initialisation if
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   263
            # needed (for instance looking for persistent configuration using an
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   264
            # internal session, which is not possible until pools have been
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   265
            # initialized)
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   266
            source.init(True, sourceent)
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   267
            if not source.copy_based_source:
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   268
                self.sources.append(source)
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   269
                self.querier.set_planner()
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   270
                if add_to_pools:
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   271
                    for pool in self.pools:
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   272
                        pool.add_source(source)
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
   273
        else:
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   274
            source.init(False, sourceent)
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   275
        self._clear_planning_caches()
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   276
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   277
    def remove_source(self, uri):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   278
        source = self.sources_by_uri.pop(uri)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   279
        del self.sources_by_eid[source.eid]
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   280
        if self.config.source_enabled(source) and not source.copy_based_source:
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   281
            self.sources.remove(source)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   282
            self.querier.set_planner()
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   283
            for pool in self.pools:
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   284
                pool.remove_source(source)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   285
        self._clear_planning_caches()
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   286
6945
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6941
diff changeset
   287
    def get_source(self, type, uri, source_config, eid=None):
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   288
        # set uri and type in source config so it's available through
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   289
        # source_defs()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   290
        source_config['uri'] = uri
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   291
        source_config['type'] = type
6945
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6941
diff changeset
   292
        return sources.get_source(type, source_config, self, eid)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   293
2963
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2961
diff changeset
   294
    def set_schema(self, schema, resetvreg=True, rebuildinfered=True):
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2961
diff changeset
   295
        if rebuildinfered:
12ad88615a12 test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2961
diff changeset
   296
            schema.rebuild_infered_relations()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   297
        self.info('set schema %s %#x', schema.name, id(schema))
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3042
diff changeset
   298
        if resetvreg:
2881
d1a5b77e42bc fix set_schema/init_cubes order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2880
diff changeset
   299
            if self.config._cubes is None:
d1a5b77e42bc fix set_schema/init_cubes order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2880
diff changeset
   300
                self.config.init_cubes(self.get_cubes())
5447
f29dafba250a [vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5442
diff changeset
   301
            # trigger full reload of all appobjects
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3042
diff changeset
   302
            self.vreg.set_schema(schema)
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3042
diff changeset
   303
        else:
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3042
diff changeset
   304
            self.vreg._set_schema(schema)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   305
        self.querier.set_schema(schema)
6126
aca6a2c357fd [repository] enabled sources refactoring: to avoid error, we should always have all known source in repo.sources_by_uri and only enabled ones in repo.sources, so we still have access to the definition of temporarily disabled sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6061
diff changeset
   306
        # don't use self.sources, we may want to give schema even to disabled
aca6a2c357fd [repository] enabled sources refactoring: to avoid error, we should always have all known source in repo.sources_by_uri and only enabled ones in repo.sources, so we still have access to the definition of temporarily disabled sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6061
diff changeset
   307
        # sources
aca6a2c357fd [repository] enabled sources refactoring: to avoid error, we should always have all known source in repo.sources_by_uri and only enabled ones in repo.sources, so we still have access to the definition of temporarily disabled sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6061
diff changeset
   308
        for source in self.sources_by_uri.values():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   309
            source.set_schema(schema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   310
        self.schema = schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   311
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   312
    def fill_schema(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   313
        """lod schema from the repository"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   314
        from cubicweb.server.schemaserial import deserialize_schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   315
        self.info('loading schema from the repository')
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   316
        appschema = schema.CubicWebSchema(self.config.appid)
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
   317
        self.set_schema(self.config.load_bootstrap_schema(), resetvreg=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   318
        self.debug('deserializing db schema into %s %#x', appschema.name, id(appschema))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   319
        session = self.internal_session()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   320
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   321
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   322
                deserialize_schema(appschema, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   323
            except BadSchemaDefinition:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   324
                raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   325
            except Exception, ex:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   326
                import traceback
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   327
                traceback.print_exc()
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   328
                raise Exception('Is the database initialised ? (cause: %s)' %
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
                                (ex.args and ex.args[0].strip() or 'unknown')), \
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   330
                                None, sys.exc_info()[-1]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   331
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   332
            session.close()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   333
        self.set_schema(appschema)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   334
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   335
    def start_looping_tasks(self):
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   336
        if not (self.config.creating or self.config.repairing
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   337
                or self.config.quick_start):
4958
665eacdd8c50 [repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   338
            # call instance level initialisation hooks
665eacdd8c50 [repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   339
            self.hm.call_hooks('server_startup', repo=self)
665eacdd8c50 [repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   340
            # register a task to cleanup expired session
6012
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   341
            self.cleanup_session_time = self.config['cleanup-session-time'] or 60 * 60 * 24
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   342
            assert self.cleanup_session_time > 0
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   343
            cleanup_session_interval = min(60*60, self.cleanup_session_time / 3)
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   344
            self.looping_task(cleanup_session_interval, self.clean_sessions)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
        assert isinstance(self._looping_tasks, list), 'already started'
2708
60d728bdcba5 allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2667
diff changeset
   346
        for i, (interval, func, args) in enumerate(self._looping_tasks):
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   347
            self._looping_tasks[i] = task = utils.LoopTask(interval, func, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   348
            self.info('starting task %s with interval %.2fs', task.name,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
                      interval)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   350
            task.start()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   351
        # ensure no tasks will be further added
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   352
        self._looping_tasks = tuple(self._looping_tasks)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   353
2708
60d728bdcba5 allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2667
diff changeset
   354
    def looping_task(self, interval, func, *args):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   355
        """register a function to be called every `interval` seconds.
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   356
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   357
        looping tasks can only be registered during repository initialization,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   358
        once done this method will fail.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   359
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   360
        try:
2708
60d728bdcba5 allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2667
diff changeset
   361
            self._looping_tasks.append( (interval, func, args) )
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   362
        except AttributeError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   363
            raise RuntimeError("can't add looping task once the repository is started")
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   364
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   365
    def threaded_task(self, func):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   366
        """start function in a separated thread"""
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   367
        t = utils.RepoThread(func, self._running_threads)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   368
        t.start()
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   369
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   370
    #@locked
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   371
    def _get_pool(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   372
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   373
            return self._available_pools.get(True, timeout=5)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   374
        except Queue.Empty:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   375
            raise Exception('no pool available after 5 secs, probably either a '
4706
6035e96b64dd added stats for munin collecting #615844 - from 027bbff3659f
arthur
parents: 4689
diff changeset
   376
                            'bug in code (too many uncommited/rollbacked '
6035e96b64dd added stats for munin collecting #615844 - from 027bbff3659f
arthur
parents: 4689
diff changeset
   377
                            'connections) or too much load on the server (in '
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   378
                            'which case you can try to set a bigger '
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   379
                            'connections pools size)')
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   380
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   381
    def _free_pool(self, pool):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   382
        self._available_pools.put_nowait(pool)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   383
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   384
    def pinfo(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   385
        # XXX: session.pool is accessed from a local storage, would be interesting
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   386
        #      to see if there is a pool set in any thread specific data)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   387
        return '%s: %s (%s)' % (self._available_pools.qsize(),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   388
                                ','.join(session.user.login for session in self._sessions.values()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   389
                                         if session.pool),
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   390
                                threading.currentThread())
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   391
    def shutdown(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   392
        """called on server stop event to properly close opened sessions and
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   393
        connections
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   394
        """
5906
d40ced753291 [repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5813
diff changeset
   395
        assert not self.shutting_down, 'already shutting down'
d40ced753291 [repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5813
diff changeset
   396
        self.shutting_down = True
5749
b4393b681f7a [repo] on repository shutdown, we've to close the new eid creation connection
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5747
diff changeset
   397
        self.system_source.shutdown()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   398
        if isinstance(self._looping_tasks, tuple): # if tasks have been started
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   399
            for looptask in self._looping_tasks:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   400
                self.info('canceling task %s...', looptask.name)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
                looptask.cancel()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   402
                looptask.join()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   403
                self.info('task %s finished', looptask.name)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   404
        for thread in self._running_threads:
5376
2c3f14bc2590 [python2.6] don't add a name property on Thread
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5292
diff changeset
   405
            self.info('waiting thread %s...', thread.getName())
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   406
            thread.join()
5376
2c3f14bc2590 [python2.6] don't add a name property on Thread
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5292
diff changeset
   407
            self.info('thread %s finished', thread.getName())
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   408
        if not (self.config.creating or self.config.repairing
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   409
                or self.config.quick_start):
4946
2654cd5c87e0 [repo] don't call server_shutdown hooks when creating/repairing instance to be consistence with server_startup hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4862
diff changeset
   410
            self.hm.call_hooks('server_shutdown', repo=self)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   411
        self.close_sessions()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   412
        while not self._available_pools.empty():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   413
            pool = self._available_pools.get_nowait()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   414
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   415
                pool.close(True)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   416
            except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
                self.exception('error while closing %s' % pool)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   418
                continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   419
        if self.pyro_registered:
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
   420
            if self._use_pyrons():
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
   421
                pyro_unregister(self.config)
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
   422
            self.pyro_uri = None
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
        hits, misses = self.querier.cache_hit, self.querier.cache_miss
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   424
        try:
5747
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   425
            self.info('rql st cache hit/miss: %s/%s (%s%% hits)', hits, misses,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   426
                      (hits * 100) / (hits + misses))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   427
            hits, misses = self.system_source.cache_hit, self.system_source.cache_miss
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   428
            self.info('sql cache hit/miss: %s/%s (%s%% hits)', hits, misses,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   429
                      (hits * 100) / (hits + misses))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   430
            nocache  = self.system_source.no_cache
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   431
            self.info('sql cache usage: %s/%s (%s%%)', hits+ misses, nocache,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   432
                      ((hits + misses) * 100) / (hits + misses + nocache))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   433
        except ZeroDivisionError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   434
            pass
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   435
6849
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   436
    def check_auth_info(self, session, login, authinfo):
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   437
        """validate authentication, raise AuthenticationError on failure, return
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   438
        associated CWUser's eid on success.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   439
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   440
        for source in self.sources:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   441
            if source.support_entity('CWUser'):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   442
                try:
6849
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   443
                    return source.authenticate(session, login, **authinfo)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   444
                except AuthenticationError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   445
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   446
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   447
            raise AuthenticationError('authentication failed with all sources')
6849
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   448
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   449
    def authenticate_user(self, session, login, **authinfo):
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   450
        """validate login / password, raise AuthenticationError on failure
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   451
        return associated CWUser instance on success
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   452
        """
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   453
        eid = self.check_auth_info(session, login, authinfo)
2268
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   454
        cwuser = self._build_user(session, eid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   455
        if self.config.consider_user_state and \
5556
9ab2b4c74baf [entity] introduce a new 'adapters' registry
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5447
diff changeset
   456
               not cwuser.cw_adapt_to('IWorkflowable').state in cwuser.AUTHENTICABLE_STATES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   457
            raise AuthenticationError('user is not in authenticable state')
2268
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   458
        return cwuser
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   459
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   460
    def _build_user(self, session, eid):
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   461
        """return a CWUser entity for user with the given eid"""
2650
18aec79ec3a3 R [vreg] important refactoring of the vregistry, moving behaviour to end dictionnary (and so leaving room for more flexibility ; keep bw compat ; update api usage in cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2647
diff changeset
   462
        cls = self.vreg['etypes'].etype_class('CWUser')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   463
        rql = cls.fetch_rql(session.user, ['X eid %(x)s'])
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
   464
        rset = session.execute(rql, {'x': eid})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   465
        assert len(rset) == 1, rset
2268
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   466
        cwuser = rset.get_entity(0, 0)
6491
ee9a10b6620e pylint option update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6467
diff changeset
   467
        # pylint: disable=W0104
2268
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   468
        # prefetch / cache cwuser's groups and properties. This is especially
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   469
        # useful for internal sessions to avoid security insertions
2268
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   470
        cwuser.groups
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   471
        cwuser.properties
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   472
        return cwuser
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   473
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   474
    # public (dbapi) interface ################################################
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   475
5587
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   476
    def stats(self): # XXX restrict to managers session?
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   477
        """Return a dictionary containing some statistics about the repository
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   478
        resources usage.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   479
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   480
        This is a public method, not requiring a session id.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   481
        """
5587
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   482
        results = {}
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   483
        querier = self.querier
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   484
        source = self.system_source
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   485
        for size, maxsize, hits, misses, title in (
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   486
            (len(querier._rql_cache), self.config['rql-cache-size'],
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   487
            querier.cache_hit, querier.cache_miss, 'rqlt_st'),
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   488
            (len(source._cache), self.config['rql-cache-size'],
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   489
            source.cache_hit, source.cache_miss, 'sql'),
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   490
            ):
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   491
            results['%s_cache_size' % title] =  '%s / %s' % (size, maxsize)
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   492
            results['%s_cache_hit' % title] =  hits
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   493
            results['%s_cache_miss' % title] = misses
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   494
            results['%s_cache_hit_percent' % title] = (hits * 100) / (hits + misses)
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   495
        results['sql_no_cache'] = self.system_source.no_cache
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   496
        results['nb_open_sessions'] = len(self._sessions)
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   497
        results['nb_active_threads'] = threading.activeCount()
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   498
        results['looping_tasks'] = ', '.join(str(t) for t in self._looping_tasks)
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   499
        results['available_pools'] = self._available_pools.qsize()
5606
61b28589d33f merge back to stable some changes made on site for a customer.
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5587 5603
diff changeset
   500
        results['threads'] = ', '.join(sorted(str(t) for t in threading.enumerate()))
5587
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   501
        return results
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   502
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   503
    def get_schema(self):
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   504
        """Return the instance schema.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   505
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   506
        This is a public method, not requiring a session id.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   507
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   508
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   509
            # necessary to support pickling used by pyro
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   510
            self.schema.__hashmode__ = 'pickle'
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   511
            return self.schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   512
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   513
            self.schema.__hashmode__ = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   514
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   515
    def get_cubes(self):
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   516
        """Return the list of cubes used by this instance.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   517
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   518
        This is a public method, not requiring a session id.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   519
        """
2473
490f88fb99b6 new distinguish repairing/creating from regular start.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2268
diff changeset
   520
        versions = self.get_versions(not (self.config.creating
4689
4eb1f4490538 [test] skipping versions checking during test is enough, no need for monkey patch
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4687
diff changeset
   521
                                          or self.config.repairing
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   522
                                          or self.config.quick_start
4689
4eb1f4490538 [test] skipping versions checking during test is enough, no need for monkey patch
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4687
diff changeset
   523
                                          or self.config.mode == 'test'))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   524
        cubes = list(versions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   525
        cubes.remove('cubicweb')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   526
        return cubes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   527
6308
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   528
    def get_option_value(self, option, foreid=None):
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   529
        """Return the value for `option` in the configuration. If `foreid` is
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   530
        specified, the actual repository to which this entity belongs is
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   531
        derefenced and the option value retrieved from it.
6257
7eb5f1aed728 [repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6256
diff changeset
   532
7eb5f1aed728 [repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6256
diff changeset
   533
        This is a public method, not requiring a session id.
7eb5f1aed728 [repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6256
diff changeset
   534
        """
7eb5f1aed728 [repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6256
diff changeset
   535
        # XXX we may want to check we don't give sensible information
6308
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   536
        if foreid is None:
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   537
            return self.config[option]
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   538
        _, sourceuri, extid = self.type_and_source_from_eid(foreid)
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   539
        if sourceuri == 'system':
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   540
            return self.config[option]
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   541
        pool = self._get_pool()
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   542
        try:
7064
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   543
            cnx = pool.connection(sourceuri)
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   544
            # needed to check connection is valid and usable by the current
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   545
            # thread
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   546
            newcnx = self.sources_by_uri[sourceuri].check_connection(cnx)
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   547
            if newcnx is not None:
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   548
                cnx = newcnx
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   549
            return cnx.get_option_value(option, extid)
6308
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   550
        finally:
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   551
            self._free_pool(pool)
6257
7eb5f1aed728 [repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6256
diff changeset
   552
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   553
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   554
    def get_versions(self, checkversions=False):
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   555
        """Return the a dictionary containing cubes used by this instance
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   556
        as key with their version as value, including cubicweb version.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   557
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   558
        This is a public method, not requiring a session id.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   559
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   560
        from logilab.common.changelog import Version
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   561
        vcconf = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   562
        session = self.internal_session()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   563
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   564
            for pk, version in session.execute(
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   565
                'Any K,V WHERE P is CWProperty, P value V, P pkey K, '
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   566
                'P pkey ~="system.version.%"', build_descr=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   567
                cube = pk.split('.')[-1]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   568
                # XXX cubicweb migration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   569
                if cube in CW_MIGRATION_MAP:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   570
                    cube = CW_MIGRATION_MAP[cube]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   571
                version = Version(version)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   572
                vcconf[cube] = version
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   573
                if checkversions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   574
                    if cube != 'cubicweb':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   575
                        fsversion = self.config.cube_version(cube)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   576
                    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   577
                        fsversion = self.config.cubicweb_version()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   578
                    if version < fsversion:
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2473
diff changeset
   579
                        msg = ('instance has %s version %s but %s '
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   580
                               'is installed. Run "cubicweb-ctl upgrade".')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   581
                        raise ExecutionError(msg % (cube, version, fsversion))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   582
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   583
            session.close()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   584
        return vcconf
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   585
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   586
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   587
    def source_defs(self):
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   588
        """Return the a dictionary containing source uris as value and a
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   589
        dictionary describing each source as value.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   590
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   591
        This is a public method, not requiring a session id.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   592
        """
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   593
        sources = {}
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   594
        # remove sensitive information
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   595
        for uri, source in self.sources_by_uri.iteritems():
6722
3341521d857b [repo source] rename attribute so it's much easier to grasp its role
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6674
diff changeset
   596
            sources[uri] = source.public_config
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   597
        return sources
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   598
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   599
    def properties(self):
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   600
        """Return a result set containing system wide properties.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   601
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   602
        This is a public method, not requiring a session id.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   603
        """
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   604
        session = self.internal_session()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   605
        try:
5747
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   606
            # don't use session.execute, we don't want rset.req set
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   607
            return self.querier.execute(session, 'Any K,V WHERE P is CWProperty,'
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   608
                                        'P pkey K, P value V, NOT P for_user U',
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   609
                                        build_descr=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   610
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   611
            session.close()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   612
5032
e3fa27fc0d9a cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5030
diff changeset
   613
    # XXX protect this method: anonymous should be allowed and registration
e3fa27fc0d9a cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5030
diff changeset
   614
    # plugged
1372
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   615
    def register_user(self, login, password, email=None, **kwargs):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   616
        """check a user with the given login exists, if not create it with the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   617
        given password. This method is designed to be used for anonymous
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   618
        registration on public web site.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   619
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   620
        session = self.internal_session()
1372
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   621
        # for consistency, keep same error as unique check hook (although not required)
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   622
        errmsg = session._('the value "%s" is already used, use another one')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   623
        try:
5073
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   624
            if (session.execute('CWUser X WHERE X login %(login)s', {'login': login},
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   625
                                build_descr=False)
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   626
                or session.execute('CWUser X WHERE X use_email C, C address %(login)s',
5073
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   627
                                   {'login': login}, build_descr=False)):
5030
5238d9a8dfee [form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4984
diff changeset
   628
                qname = role_name('login', 'subject')
5238d9a8dfee [form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4984
diff changeset
   629
                raise ValidationError(None, {qname: errmsg % login})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   630
            # we have to create the user
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
   631
            user = self.vreg['etypes'].etype_class('CWUser')(session)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   632
            if isinstance(password, unicode):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   633
                # password should *always* be utf8 encoded
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   634
                password = password.encode('UTF8')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   635
            kwargs['login'] = login
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   636
            kwargs['upassword'] = password
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
   637
            self.glob_add_entity(session, EditedEntity(user, **kwargs))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   638
            session.execute('SET X in_group G WHERE X eid %(x)s, G name "users"',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   639
                            {'x': user.eid})
1372
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   640
            if email or '@' in login:
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   641
                d = {'login': login, 'email': email or login}
5073
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   642
                if session.execute('EmailAddress X WHERE X address %(email)s', d,
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   643
                                   build_descr=False):
5030
5238d9a8dfee [form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4984
diff changeset
   644
                    qname = role_name('address', 'subject')
5238d9a8dfee [form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4984
diff changeset
   645
                    raise ValidationError(None, {qname: errmsg % d['email']})
1372
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   646
                session.execute('INSERT EmailAddress X: X address %(email)s, '
5032
e3fa27fc0d9a cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5030
diff changeset
   647
                                'U primary_email X, U use_email X '
5073
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   648
                                'WHERE U login %(login)s', d, build_descr=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   649
            session.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   650
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   651
            session.close()
594
76218d42d21f return success or not on creation of user
Arthur Lutz <arthur.lutz@logilab.fr>
parents: 479
diff changeset
   652
        return True
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   653
6390
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   654
    def find_users(self, fetch_attrs, **query_attrs):
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   655
        """yield user attributes for cwusers matching the given query_attrs
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   656
        (the result set cannot survive this method call)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   657
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   658
        This can be used by low-privileges account (anonymous comes to
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   659
        mind).
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   660
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   661
        `fetch_attrs`: tuple of attributes to be fetched
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   662
        `query_attrs`: dict of attr/values to restrict the query
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   663
        """
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   664
        assert query_attrs
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   665
        if not hasattr(self, '_cwuser_attrs'):
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   666
            cwuser = self.schema['CWUser']
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   667
            self._cwuser_attrs = set(str(rschema)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   668
                                     for rschema, _eschema in cwuser.attribute_definitions()
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   669
                                     if not rschema.meta)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   670
        cwuserattrs = self._cwuser_attrs
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   671
        for k in chain(fetch_attrs, query_attrs.iterkeys()):
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   672
            if k not in cwuserattrs:
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   673
                raise Exception('bad input for find_user')
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   674
        session = self.internal_session()
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   675
        try:
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   676
            varmaker = rqlvar_maker()
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   677
            vars = [(attr, varmaker.next()) for attr in fetch_attrs]
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   678
            rql = 'Any %s WHERE X is CWUser, ' % ','.join(var[1] for var in vars)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   679
            rql += ','.join('X %s %s' % (var[0], var[1]) for var in vars) + ','
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   680
            rset = session.execute(rql + ','.join('X %s %%(%s)s' % (attr, attr)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   681
                                                  for attr in query_attrs.iterkeys()),
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   682
                                   query_attrs)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   683
            return rset.rows
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   684
        finally:
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   685
            session.close()
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   686
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3629
diff changeset
   687
    def connect(self, login, **kwargs):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   688
        """open a connection for a given user
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   689
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   690
        base_url may be needed to send mails
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   691
        cnxtype indicate if this is a pyro connection or a in-memory connection
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   692
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   693
        raise `AuthenticationError` if the authentication failed
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   694
        raise `ConnectionError` if we can't open a connection
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   695
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   696
        # use an internal connection
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   697
        session = self.internal_session()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   698
        # try to get a user object
4089
ff92c7d692bf typos, api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3890
diff changeset
   699
        cnxprops = kwargs.pop('cnxprops', None)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   700
        try:
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3629
diff changeset
   701
            user = self.authenticate_user(session, login, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   702
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   703
            session.close()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   704
        session = Session(user, self, cnxprops)
3379
9192ba07890d use .cw_rset instead of rset on appobject classes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3293
diff changeset
   705
        user._cw = user.cw_rset.req = session
5557
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   706
        user.cw_clear_relation_cache()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   707
        self._sessions[session.id] = session
5638
cea831403e72 log the user name on session opening for easier debugging
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5603
diff changeset
   708
        self.info('opened session %s for user %s', session.id, login)
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
   709
        self.hm.call_hooks('session_open', session)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   710
        # commit session at this point in case write operation has been done
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   711
        # during `session_open` hooks
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   712
        session.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   713
        return session.id
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   714
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   715
    def execute(self, sessionid, rqlstring, args=None, build_descr=True,
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   716
                txid=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   717
        """execute a RQL query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   718
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   719
        * rqlstring should be an unicode string or a plain ascii string
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   720
        * args the optional parameters used in the query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   721
        * build_descr is a flag indicating if the description should be
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   722
          built on select queries
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   723
        """
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   724
        session = self._get_session(sessionid, setpool=True, txid=txid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   725
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   726
            try:
5747
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   727
                rset = self.querier.execute(session, rqlstring, args,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   728
                                            build_descr)
5747
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   729
                # NOTE: the web front will (re)build it when needed
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   730
                #       e.g in facets
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   731
                #       Zeroed to avoid useless overhead with pyro
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   732
                rset._rqlst = None
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   733
                return rset
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   734
            except (Unauthorized, RQLSyntaxError):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   735
                raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   736
            except ValidationError, ex:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   737
                # need ValidationError normalization here so error may pass
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   738
                # through pyro
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   739
                if hasattr(ex.entity, 'eid'):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   740
                    ex.entity = ex.entity.eid # error raised by yams
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   741
                    args = list(ex.args)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   742
                    args[0] = ex.entity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   743
                    ex.args = tuple(args)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   744
                raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   745
            except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   746
                # FIXME: check error to catch internal errors
5167
529861a73ec8 log RQL query in case of unexpected failure (priceless when porting to new database)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5115
diff changeset
   747
                self.exception('unexpected error while executing %s with %s', rqlstring, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   748
                raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   749
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   750
            session.reset_pool()
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   751
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   752
    def describe(self, sessionid, eid, txid=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   753
        """return a tuple (type, source, extid) for the entity with id <eid>"""
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   754
        session = self._get_session(sessionid, setpool=True, txid=txid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   755
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   756
            return self.type_and_source_from_eid(eid, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   757
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   758
            session.reset_pool()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   759
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   760
    def check_session(self, sessionid):
6012
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   761
        """raise `BadConnectionId` if the connection is no more valid, else
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   762
        return its latest activity timestamp.
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   763
        """
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   764
        return self._get_session(sessionid, setpool=False).timestamp
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   765
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   766
    def get_shared_data(self, sessionid, key, default=None, pop=False, txdata=False):
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   767
        """return value associated to key in the session's data dictionary or
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   768
        session's transaction's data if `txdata` is true.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   769
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   770
        If pop is True, value will be removed from the dictionnary.
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   771
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   772
        If key isn't defined in the dictionnary, value specified by the
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   773
        `default` argument will be returned.
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   774
        """
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   775
        session = self._get_session(sessionid, setpool=False)
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   776
        return session.get_shared_data(key, default, pop, txdata)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   777
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   778
    def set_shared_data(self, sessionid, key, value, txdata=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   779
        """set value associated to `key` in shared data
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   780
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   781
        if `txdata` is true, the value will be added to the repository session's
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   782
        transaction's data which are cleared on commit/rollback of the current
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   783
        transaction.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   784
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   785
        session = self._get_session(sessionid, setpool=False)
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   786
        session.set_shared_data(key, value, txdata)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   787
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   788
    def commit(self, sessionid, txid=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   789
        """commit transaction for the session with the given id"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   790
        self.debug('begin commit for session %s', sessionid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   791
        try:
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   792
            session = self._get_session(sessionid)
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   793
            session.set_tx_data(txid)
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   794
            return session.commit()
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   795
        except (ValidationError, Unauthorized):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   796
            raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   797
        except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   798
            self.exception('unexpected error')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   799
            raise
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   800
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   801
    def rollback(self, sessionid, txid=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   802
        """commit transaction for the session with the given id"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   803
        self.debug('begin rollback for session %s', sessionid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   804
        try:
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   805
            session = self._get_session(sessionid)
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   806
            session.set_tx_data(txid)
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   807
            session.rollback()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   808
        except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   809
            self.exception('unexpected error')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   810
            raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   811
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   812
    def close(self, sessionid, txid=None, checkshuttingdown=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   813
        """close the session with the given id"""
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   814
        session = self._get_session(sessionid, setpool=True, txid=txid,
1939
67e7379edd96 #343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1883
diff changeset
   815
                                    checkshuttingdown=checkshuttingdown)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   816
        # operation uncommited before close are rollbacked before hook is called
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   817
        session.rollback(reset_pool=False)
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
   818
        self.hm.call_hooks('session_close', session)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   819
        # commit session at this point in case write operation has been done
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   820
        # during `session_close` hooks
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   821
        session.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   822
        session.close()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   823
        del self._sessions[sessionid]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   824
        self.info('closed session %s for user %s', sessionid, session.user.login)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   825
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   826
    def user_info(self, sessionid, props=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   827
        """this method should be used by client to:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   828
        * check session id validity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   829
        * update user information on each user's request (i.e. groups and
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   830
          custom properties)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   831
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   832
        session = self._get_session(sessionid, setpool=False)
2245
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   833
        if props is not None:
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   834
            self.set_session_props(sessionid, props)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   835
        user = session.user
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   836
        return user.eid, user.login, user.groups, user.properties
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   837
2245
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   838
    def set_session_props(self, sessionid, props):
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   839
        """this method should be used by client to:
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   840
        * check session id validity
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   841
        * update user information on each user's request (i.e. groups and
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   842
          custom properties)
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   843
        """
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   844
        session = self._get_session(sessionid, setpool=False)
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   845
        for prop, value in props.items():
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   846
            session.change_property(prop, value)
7463e1a748dd new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2192
diff changeset
   847
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   848
    def undoable_transactions(self, sessionid, ueid=None, txid=None,
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   849
                              **actionfilters):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   850
        """See :class:`cubicweb.dbapi.Connection.undoable_transactions`"""
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   851
        session = self._get_session(sessionid, setpool=True, txid=txid)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   852
        try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   853
            return self.system_source.undoable_transactions(session, ueid,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   854
                                                            **actionfilters)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   855
        finally:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   856
            session.reset_pool()
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   857
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   858
    def transaction_info(self, sessionid, txuuid, txid=None):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   859
        """See :class:`cubicweb.dbapi.Connection.transaction_info`"""
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   860
        session = self._get_session(sessionid, setpool=True, txid=txid)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   861
        try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   862
            return self.system_source.tx_info(session, txuuid)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   863
        finally:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   864
            session.reset_pool()
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   865
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   866
    def transaction_actions(self, sessionid, txuuid, public=True, txid=None):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   867
        """See :class:`cubicweb.dbapi.Connection.transaction_actions`"""
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   868
        session = self._get_session(sessionid, setpool=True, txid=txid)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   869
        try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   870
            return self.system_source.tx_actions(session, txuuid, public)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   871
        finally:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   872
            session.reset_pool()
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   873
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   874
    def undo_transaction(self, sessionid, txuuid, txid=None):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   875
        """See :class:`cubicweb.dbapi.Connection.undo_transaction`"""
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   876
        session = self._get_session(sessionid, setpool=True, txid=txid)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   877
        try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   878
            return self.system_source.undo_transaction(session, txuuid)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   879
        finally:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   880
            session.reset_pool()
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   881
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   882
    # public (inter-repository) interface #####################################
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   883
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   884
    def entities_modified_since(self, etypes, mtime):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   885
        """function designed to be called from an external repository which
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   886
        is using this one as a rql source for synchronization, and return a
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   887
        3-uple containing :
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   888
        * the local date
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   889
        * list of (etype, eid) of entities of the given types which have been
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   890
          modified since the given timestamp (actually entities whose full text
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   891
          index content has changed)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   892
        * list of (etype, eid) of entities of the given types which have been
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   893
          deleted since the given timestamp
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   894
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   895
        session = self.internal_session()
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 636
diff changeset
   896
        updatetime = datetime.now()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   897
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   898
            modentities, delentities = self.system_source.modified_entities(
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   899
                session, etypes, mtime)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   900
            return updatetime, modentities, delentities
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   901
        finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   902
            session.close()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   903
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   904
    # session handling ########################################################
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   905
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   906
    def close_sessions(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   907
        """close every opened sessions"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   908
        for sessionid in self._sessions.keys():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   909
            try:
1939
67e7379edd96 #343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1883
diff changeset
   910
                self.close(sessionid, checkshuttingdown=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   911
            except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   912
                self.exception('error while closing session %s' % sessionid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   913
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   914
    def clean_sessions(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   915
        """close sessions not used since an amount of time specified in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   916
        configuration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   917
        """
6012
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   918
        mintime = time() - self.cleanup_session_time
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   919
        self.debug('cleaning session unused since %s',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   920
                   strftime('%T', localtime(mintime)))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   921
        nbclosed = 0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   922
        for session in self._sessions.values():
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   923
            if session.timestamp < mintime:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   924
                self.close(session.id)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   925
                nbclosed += 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   926
        return nbclosed
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   927
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   928
    def internal_session(self, cnxprops=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   929
        """return a dbapi like connection/cursor using internal user which
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   930
        have every rights on the repository. You'll *have to* commit/rollback
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   931
        or close (rollback implicitly) the session once the job's done, else
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   932
        you'll leak connections pool up to the time where no more pool is
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   933
        available, causing irremediable freeze...
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   934
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   935
        session = InternalSession(self, cnxprops)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   936
        session.set_pool()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   937
        return session
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   938
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   939
    def _get_session(self, sessionid, setpool=False, txid=None,
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   940
                     checkshuttingdown=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   941
        """return the user associated to the given session identifier"""
5906
d40ced753291 [repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5813
diff changeset
   942
        if checkshuttingdown and self.shutting_down:
1883
011e13d74cfc shuting -> shutting
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 1880
diff changeset
   943
            raise Exception('Repository is shutting down')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   944
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   945
            session = self._sessions[sessionid]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   946
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   947
            raise BadConnectionId('No such session %s' % sessionid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   948
        if setpool:
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   949
            session.set_tx_data(txid) # must be done before set_pool
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   950
            session.set_pool()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   951
        return session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   952
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   953
    # data sources handling ###################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   954
    # * correspondance between eid and (type, source)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   955
    # * correspondance between eid and local id (i.e. specific to a given source)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   956
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   957
    def type_and_source_from_eid(self, eid, session=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   958
        """return a tuple (type, source, extid) for the entity with id <eid>"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   959
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   960
            eid = typed_eid(eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   961
        except ValueError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   962
            raise UnknownEid(eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   963
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   964
            return self._type_source_cache[eid]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   965
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   966
            if session is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   967
                session = self.internal_session()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   968
                reset_pool = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   969
            else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   970
                reset_pool = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   971
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   972
                etype, uri, extid = self.system_source.eid_type_source(session,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   973
                                                                       eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   974
            finally:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   975
                if reset_pool:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   976
                    session.reset_pool()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   977
        self._type_source_cache[eid] = (etype, uri, extid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   978
        if uri != 'system':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   979
            self._extid_cache[(extid, uri)] = eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   980
        return etype, uri, extid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   981
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   982
    def clear_caches(self, eids):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   983
        etcache = self._type_source_cache
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   984
        extidcache = self._extid_cache
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   985
        rqlcache = self.querier._rql_cache
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   986
        for eid in eids:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   987
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   988
                etype, uri, extid = etcache.pop(typed_eid(eid)) # may be a string in some cases
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   989
                rqlcache.pop('%s X WHERE X eid %s' % (etype, eid), None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   990
                extidcache.pop((extid, uri), None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   991
            except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   992
                etype = None
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   993
            rqlcache.pop('Any X WHERE X eid %s' % eid, None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   994
            for source in self.sources:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   995
                source.clear_eid_cache(eid, etype)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   996
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   997
    def type_from_eid(self, eid, session=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   998
        """return the type of the entity with id <eid>"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   999
        return self.type_and_source_from_eid(eid, session)[0]
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1000
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1001
    def source_from_eid(self, eid, session=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1002
        """return the source for the given entity's eid"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1003
        return self.sources_by_uri[self.type_and_source_from_eid(eid, session)[1]]
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1004
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1005
    def querier_cache_key(self, session, rql, args, eidkeys):
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1006
        cachekey = [rql]
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1007
        for key in sorted(eidkeys):
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1008
            try:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1009
                etype = self.type_from_eid(args[key], session)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1010
            except KeyError:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1011
                raise QueryError('bad cache key %s (no value)' % key)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1012
            except TypeError:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1013
                raise QueryError('bad cache key %s (value: %r)' % (
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1014
                    key, args[key]))
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1015
            cachekey.append(etype)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1016
            # ensure eid is correctly typed in args
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1017
            args[key] = typed_eid(args[key])
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1018
        return tuple(cachekey)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1019
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1020
    def eid2extid(self, source, eid, session=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1021
        """get local id from an eid"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1022
        etype, uri, extid = self.type_and_source_from_eid(eid, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1023
        if source.uri != uri:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1024
            # eid not from the given source
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1025
            raise UnknownEid(eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1026
        return extid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1027
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1028
    def extid2eid(self, source, extid, etype, session=None, insert=True,
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1029
                  sourceparams=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1030
        """get eid from a local id. An eid is attributed if no record is found"""
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1031
        uri = 'system' if source.copy_based_source else source.uri
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1032
        cachekey = (extid, uri)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1033
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1034
            return self._extid_cache[cachekey]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1035
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1036
            pass
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1037
        reset_pool = False
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1038
        if session is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1039
            session = self.internal_session()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1040
            reset_pool = True
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1041
        eid = self.system_source.extid2eid(session, uri, extid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1042
        if eid is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1043
            self._extid_cache[cachekey] = eid
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1044
            self._type_source_cache[eid] = (etype, uri, extid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1045
            if reset_pool:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1046
                session.reset_pool()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1047
            return eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1048
        if not insert:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1049
            return
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1050
        # no link between extid and eid, create one using an internal session
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1051
        # since the current session user may not have required permissions to
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1052
        # do necessary stuff and we don't want to commit user session.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1053
        #
450
5e14ea0e81c8 a note for later
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 341
diff changeset
  1054
        # Moreover, even if session is already an internal session but is
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1055
        # processing a commit, we have to use another one
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1056
        if not session.is_internal_session:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1057
            session = self.internal_session()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1058
            reset_pool = True
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1059
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1060
            eid = self.system_source.create_eid(session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1061
            self._extid_cache[cachekey] = eid
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1062
            self._type_source_cache[eid] = (etype, uri, extid)
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1063
            entity = source.before_entity_insertion(
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1064
                session, extid, etype, eid, sourceparams)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1065
            if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1066
                self.hm.call_hooks('before_add_entity', session, entity=entity)
450
5e14ea0e81c8 a note for later
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 341
diff changeset
  1067
            # XXX call add_info with complete=False ?
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1068
            self.add_info(session, entity, source, extid)
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1069
            source.after_entity_insertion(session, extid, entity, sourceparams)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1070
            if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1071
                self.hm.call_hooks('after_add_entity', session, entity=entity)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1072
            session.commit(reset_pool)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1073
            return eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1074
        except:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1075
            session.rollback(reset_pool)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1076
            raise
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1077
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1078
    def add_info(self, session, entity, source, extid=None, complete=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1079
        """add type and source info for an eid into the system table,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1080
        and index the entity with the full text index
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1081
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1082
        # begin by inserting eid/type/source/extid into the entities table
6426
541659c39f6a [hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6390
diff changeset
  1083
        hook.CleanupNewEidsCacheOp.get_instance(session).add_data(entity.eid)
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4756
diff changeset
  1084
        self.system_source.add_info(session, entity, source, extid, complete)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1085
6953
260f662be125 [repo, ms] fix pb introduced in 6941:9ed02daa7dbb resulting in missing relation deletes when an entity is deleted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
  1086
    def delete_info(self, session, entity, sourceuri, extid, scleanup=None):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1087
        """called by external source when some entity known by the system source
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1088
        has been deleted in the external source
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1089
        """
5067
adc2122eed03 [repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5043
diff changeset
  1090
        # mark eid as being deleted in session info and setup cache update
adc2122eed03 [repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5043
diff changeset
  1091
        # operation
6426
541659c39f6a [hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6390
diff changeset
  1092
        hook.CleanupDeletedEidsCacheOp.get_instance(session).add_data(entity.eid)
6626
5c20a005bddc [pyro source] when cleaning relations of an external entity being cleaned up, only delete local relationsd
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6491
diff changeset
  1093
        self._delete_info(session, entity, sourceuri, extid, scleanup)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1094
6953
260f662be125 [repo, ms] fix pb introduced in 6941:9ed02daa7dbb resulting in missing relation deletes when an entity is deleted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
  1095
    def delete_info_multi(self, session, entities, sourceuri, extids, scleanup=None):
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1096
        """same as delete_info but accepts a list of entities and
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1097
        extids with the same etype and belonging to the same source
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1098
        """
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1099
        # mark eid as being deleted in session info and setup cache update
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1100
        # operation
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1101
        op = hook.CleanupDeletedEidsCacheOp.get_instance(session)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1102
        for entity in entities:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1103
            op.add_data(entity.eid)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1104
        self._delete_info_multi(session, entities, sourceuri, extids, scleanup)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1105
6941
9ed02daa7dbb [repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
  1106
    def _delete_info(self, session, entity, sourceuri, extid, scleanup=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1107
        """delete system information on deletion of an entity:
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1108
        * delete all remaining relations from/to this entity
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1109
        * call delete info on the system source which will transfer record from
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1110
          the entities table to the deleted_entities table
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1111
        """
2641
9c33d98a074e R [schema hooks] big refactoring / reorganization for clearer code, a few fixes on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2631
diff changeset
  1112
        pendingrtypes = session.transaction_data.get('pendingrtypes', ())
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1113
        # delete remaining relations: if user can delete the entity, he can
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1114
        # delete all its relations without security checking
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1115
        with security_enabled(session, read=False, write=False):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1116
            eid = entity.eid
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1117
            for rschema, _, role in entity.e_schema.relation_definitions():
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1118
                rtype = rschema.type
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1119
                if rtype in schema.VIRTUAL_RTYPES or rtype in pendingrtypes:
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1120
                    continue
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1121
                if role == 'subject':
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1122
                    # don't skip inlined relation so they are regularly
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1123
                    # deleted and so hooks are correctly called
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
  1124
                    rql = 'DELETE X %s Y WHERE X eid %%(x)s' % rtype
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1125
                else:
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
  1126
                    rql = 'DELETE Y %s X WHERE X eid %%(x)s' % rtype
6941
9ed02daa7dbb [repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
  1127
                if scleanup is not None:
6626
5c20a005bddc [pyro source] when cleaning relations of an external entity being cleaned up, only delete local relationsd
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6491
diff changeset
  1128
                    # source cleaning: only delete relations stored locally
6941
9ed02daa7dbb [repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
  1129
                    # (here, scleanup
9ed02daa7dbb [repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
  1130
                    rql += ', NOT (Y cw_source S, S eid %(seid)s)'
6674
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1131
                try:
6941
9ed02daa7dbb [repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
  1132
                    session.execute(rql, {'x': eid, 'seid': scleanup},
6674
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1133
                                    build_descr=False)
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1134
                except:
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1135
                    self.exception('error while cascading delete for entity %s '
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1136
                                   'from %s. RQL: %s', entity, sourceuri, rql)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1137
        self.system_source.delete_info(session, entity, sourceuri, extid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1138
6953
260f662be125 [repo, ms] fix pb introduced in 6941:9ed02daa7dbb resulting in missing relation deletes when an entity is deleted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6945
diff changeset
  1139
    def _delete_info_multi(self, session, entities, sourceuri, extids, scleanup=None):
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1140
        """same as _delete_info but accepts a list of entities with
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1141
        the same etype and belinging to the same source.
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1142
        """
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1143
        pendingrtypes = session.transaction_data.get('pendingrtypes', ())
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1144
        # delete remaining relations: if user can delete the entity, he can
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1145
        # delete all its relations without security checking
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1146
        assert entities and len(entities) == len(extids)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1147
        with security_enabled(session, read=False, write=False):
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1148
            eids = [_e.eid for _e in entities]
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1149
            in_eids = ','.join((str(eid) for eid in eids))
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1150
            for rschema, _, role in entities[0].e_schema.relation_definitions():
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1151
                rtype = rschema.type
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1152
                if rtype in schema.VIRTUAL_RTYPES or rtype in pendingrtypes:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1153
                    continue
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1154
                if role == 'subject':
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1155
                    # don't skip inlined relation so they are regularly
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1156
                    # deleted and so hooks are correctly called
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1157
                    rql = 'DELETE X %s Y WHERE X eid IN (%s)' % (rtype, in_eids)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1158
                else:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1159
                    rql = 'DELETE Y %s X WHERE X eid IN (%s)' % (rtype, in_eids)
6941
9ed02daa7dbb [repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
  1160
                if scleanup is not None:
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1161
                    # source cleaning: only delete relations stored locally
6941
9ed02daa7dbb [repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
  1162
                    rql += ', NOT (Y cw_source S, S eid %(seid)s)'
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1163
                try:
6941
9ed02daa7dbb [repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
  1164
                    session.execute(rql, {'seid': scleanup}, build_descr=False)
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1165
                except:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1166
                    self.exception('error while cascading delete for entity %s '
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1167
                                   'from %s. RQL: %s', entities, sourceuri, rql)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1168
        self.system_source.delete_info_multi(session, entities, sourceuri, extids)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1169
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1170
    def locate_relation_source(self, session, subject, rtype, object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1171
        subjsource = self.source_from_eid(subject, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1172
        objsource = self.source_from_eid(object, session)
3042
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1173
        if not subjsource is objsource:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1174
            source = self.system_source
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1175
            if not (subjsource.may_cross_relation(rtype)
3042
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1176
                    and objsource.may_cross_relation(rtype)):
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1177
                raise MultiSourcesError(
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1178
                    "relation %s can't be crossed among sources"
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1179
                    % rtype)
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1180
        elif not subjsource.support_relation(rtype):
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1181
            source = self.system_source
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1182
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1183
            source = subjsource
3042
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1184
        if not source.support_relation(rtype, True):
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1185
            raise MultiSourcesError(
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1186
                "source %s doesn't support write of %s relation"
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1187
                % (source.uri, rtype))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1188
        return source
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1189
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1190
    def locate_etype_source(self, etype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1191
        for source in self.sources:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1192
            if source.support_entity(etype, 1):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1193
                return source
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1194
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1195
            raise ETypeNotSupportedBySources(etype)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1196
5068
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1197
    def init_entity_caches(self, session, entity, source):
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1198
        """add entity to session entities cache and repo's extid cache.
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1199
        Return entity's ext id if the source isn't the system source.
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1200
        """
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1201
        session.set_entity_cache(entity)
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1202
        suri = source.uri
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1203
        if suri == 'system':
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1204
            extid = None
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1205
        else:
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1206
            if source.copy_based_source:
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1207
                suri = 'system'
5068
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1208
            extid = source.get_extid(entity)
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1209
            self._extid_cache[(str(extid), suri)] = entity.eid
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1210
        self._type_source_cache[entity.eid] = (entity.__regid__, suri, extid)
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1211
        return extid
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1212
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1213
    def glob_add_entity(self, session, edited):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1214
        """add an entity to the repository
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1215
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1216
        the entity eid should originaly be None and a unique eid is assigned to
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1217
        the entity instance
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1218
        """
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1219
        entity = edited.entity
5557
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1220
        entity._cw_is_saved = False # entity has an eid but is not yet saved
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1221
        # init edited_attributes before calling before_add_entity hooks
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1222
        entity.cw_edited = edited
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1223
        eschema = entity.e_schema
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
  1224
        source = self.locate_etype_source(entity.__regid__)
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
  1225
        # allocate an eid to the entity before calling hooks
5557
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1226
        entity.eid = self.system_source.create_eid(session)
5068
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1227
        # set caches asap
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1228
        extid = self.init_entity_caches(session, entity, source)
2600
6cd6c5d11b45 [F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
  1229
        if server.DEBUG & server.DBG_REPO:
6466
43d71dd8a8ec cleanup and 3.10 updates
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6465
diff changeset
  1230
            print 'ADD entity', self, entity.__regid__, entity.eid, edited
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1231
        relations = []
2929
51cdfe069e10 fix edited_attributes handling when adding entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2875
diff changeset
  1232
        if source.should_call_hooks:
2968
0e3460341023 somewhat painful backport of 3.5 branch, should mostly be ok
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2893 2967
diff changeset
  1233
            self.hm.call_hooks('before_add_entity', session, entity=entity)
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1234
        for attr in edited.iterkeys():
3689
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3606
diff changeset
  1235
            rschema = eschema.subjrels[attr]
deb13e88e037 follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3606
diff changeset
  1236
            if not rschema.final: # inlined relation
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1237
                relations.append((attr, edited[attr]))
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1238
        edited.set_defaults()
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
  1239
        if session.is_hook_category_activated('integrity'):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1240
            edited.check(creation=True)
6464
11f9fbf6a645 [repo, entity] move entity cache initialization to a function, and call it before source.add_entity so it may be used in error handler or such
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6427
diff changeset
  1241
        prefill_entity_caches(entity, relations)
6211
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1242
        try:
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1243
            source.add_entity(session, entity)
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1244
        except UniqueTogetherError, exc:
6465
6401a9d0b5aa [architecture] introduce adapter to ease transformation of errors before display to the end user. Use it for UniqueTogetherError first
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6464
diff changeset
  1245
            userhdlr = session.vreg['adapters'].select(
6401a9d0b5aa [architecture] introduce adapter to ease transformation of errors before display to the end user. Use it for UniqueTogetherError first
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6464
diff changeset
  1246
                'IUserFriendlyError', session, entity=entity, exc=exc)
6401a9d0b5aa [architecture] introduce adapter to ease transformation of errors before display to the end user. Use it for UniqueTogetherError first
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6464
diff changeset
  1247
            userhdlr.raise_user_exception()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1248
        self.add_info(session, entity, source, extid, complete=False)
6368
f907cc7f2875 [repo] properly mark object as saved once added to its source (code much probably gone during a merge...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6333
diff changeset
  1249
        edited.saved = entity._cw_is_saved = True
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1250
        # trigger after_add_entity after after_add_relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1251
        if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1252
            self.hm.call_hooks('after_add_entity', session, entity=entity)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1253
            # call hooks for inlined relations
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1254
            for attr, value in relations:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1255
                self.hm.call_hooks('before_add_relation', session,
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1256
                                    eidfrom=entity.eid, rtype=attr, eidto=value)
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1257
                self.hm.call_hooks('after_add_relation', session,
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1258
                                    eidfrom=entity.eid, rtype=attr, eidto=value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1259
        return entity.eid
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1260
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1261
    def glob_update_entity(self, session, edited):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1262
        """replace an entity in the repository
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1263
        the type and the eid of an entity must not be changed
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1264
        """
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1265
        entity = edited.entity
2600
6cd6c5d11b45 [F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
  1266
        if server.DEBUG & server.DBG_REPO:
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
  1267
            print 'UPDATE entity', entity.__regid__, entity.eid, \
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1268
                  entity.cw_attr_cache, edited
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1269
        hm = self.hm
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1270
        eschema = entity.e_schema
2647
b0a2e779845c enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2641
diff changeset
  1271
        session.set_entity_cache(entity)
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1272
        orig_edited = getattr(entity, 'cw_edited', None)
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1273
        entity.cw_edited = edited
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1274
        try:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1275
            only_inline_rels, need_fti_update = True, False
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1276
            relations = []
5292
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1277
            source = self.source_from_eid(entity.eid, session)
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1278
            for attr in list(edited):
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1279
                if attr == 'eid':
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1280
                    continue
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1281
                rschema = eschema.subjrels[attr]
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1282
                if rschema.final:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1283
                    if getattr(eschema.rdef(attr), 'fulltextindexed', False):
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1284
                        need_fti_update = True
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1285
                    only_inline_rels = False
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1286
                else:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1287
                    # inlined relation
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1288
                    previous_value = entity.related(attr) or None
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1289
                    if previous_value is not None:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1290
                        previous_value = previous_value[0][0] # got a result set
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1291
                        if previous_value == entity.cw_attr_cache[attr]:
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1292
                            previous_value = None
5292
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1293
                        elif source.should_call_hooks:
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1294
                            hm.call_hooks('before_delete_relation', session,
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1295
                                          eidfrom=entity.eid, rtype=attr,
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1296
                                          eidto=previous_value)
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1297
                    relations.append((attr, edited[attr], previous_value))
5292
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1298
            if source.should_call_hooks:
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1299
                # call hooks for inlined relations
6211
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1300
                for attr, value, _t in relations:
5292
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1301
                    hm.call_hooks('before_add_relation', session,
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1302
                                  eidfrom=entity.eid, rtype=attr, eidto=value)
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1303
                if not only_inline_rels:
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1304
                    hm.call_hooks('before_update_entity', session, entity=entity)
5808
2de32c0c293b [repo] call entity.check() once before_update hooks has been called, to let them a chance to transform data
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5749
diff changeset
  1305
            if session.is_hook_category_activated('integrity'):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1306
                edited.check()
6211
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1307
            try:
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1308
                source.update_entity(session, entity)
6225
a176e68b7d0d backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6142 6211
diff changeset
  1309
                edited.saved = True
6211
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1310
            except UniqueTogetherError, exc:
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1311
                etype, rtypes = exc.args
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1312
                problems = {}
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1313
                for col in rtypes:
6225
a176e68b7d0d backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6142 6211
diff changeset
  1314
                    problems[col] = session._('violates unique_together constraints (%s)') % (','.join(rtypes))
6211
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1315
                raise ValidationError(entity.eid, problems)
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1316
            self.system_source.update_info(session, entity, need_fti_update)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1317
            if source.should_call_hooks:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1318
                if not only_inline_rels:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1319
                    hm.call_hooks('after_update_entity', session, entity=entity)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1320
                for attr, value, prevvalue in relations:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1321
                    # if the relation is already cached, update existant cache
5557
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1322
                    relcache = entity.cw_relation_cached(attr, 'subject')
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1323
                    if prevvalue is not None:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1324
                        hm.call_hooks('after_delete_relation', session,
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1325
                                      eidfrom=entity.eid, rtype=attr, eidto=prevvalue)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1326
                        if relcache is not None:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1327
                            session.update_rel_cache_del(entity.eid, attr, prevvalue)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1328
                    del_existing_rel_if_needed(session, entity.eid, attr, value)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1329
                    if relcache is not None:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1330
                        session.update_rel_cache_add(entity.eid, attr, value)
2647
b0a2e779845c enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2641
diff changeset
  1331
                    else:
5557
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1332
                        entity.cw_set_relation_cache(attr, 'subject',
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1333
                                                     session.eid_rset(value))
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1334
                    hm.call_hooks('after_add_relation', session,
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1335
                                  eidfrom=entity.eid, rtype=attr, eidto=value)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1336
        finally:
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1337
            if orig_edited is not None:
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1338
                entity.cw_edited = orig_edited
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1339
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1340
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1341
    def glob_delete_entities(self, session, eids):
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1342
        """delete a list of  entities and all related entities from the repository"""
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1343
        data_by_etype_source = {} # values are ([list of eids],
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1344
                                  #             [list of extid],
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1345
                                  #             [list of entities])
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1346
        #
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1347
        # WARNING: the way this dictionary is populated is heavily optimized
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1348
        # and does not use setdefault on purpose. Unless a new release
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1349
        # of the Python interpreter advertises large perf improvements
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1350
        # in setdefault, this should not be changed without profiling.
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1351
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1352
        for eid in eids:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1353
            etype, sourceuri, extid = self.type_and_source_from_eid(eid, session)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1354
            entity = session.entity_from_eid(eid, etype)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1355
            _key = (etype, sourceuri)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1356
            if _key not in data_by_etype_source:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1357
                data_by_etype_source[_key] = ([eid], [extid], [entity])
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1358
            else:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1359
                _data = data_by_etype_source[_key]
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1360
                _data[0].append(eid)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1361
                _data[1].append(extid)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1362
                _data[2].append(entity)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1363
        for (etype, sourceuri), (eids, extids, entities) in data_by_etype_source.iteritems():
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1364
            if server.DEBUG & server.DBG_REPO:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1365
                print 'DELETE entities', etype, eids
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1366
            #print 'DELETE entities', etype, len(eids)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1367
            source = self.sources_by_uri[sourceuri]
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1368
            if source.should_call_hooks:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1369
                self.hm.call_hooks('before_delete_entity', session, entities=entities)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1370
            self._delete_info_multi(session, entities, sourceuri, extids) # xxx
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1371
            source.delete_entities(session, entities)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1372
            if source.should_call_hooks:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1373
                self.hm.call_hooks('after_delete_entity', session, entities=entities)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1374
        # don't clear cache here this is done in a hook on commit
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1375
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1376
    def glob_add_relation(self, session, subject, rtype, object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1377
        """add a relation to the repository"""
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1378
        self.glob_add_relations(session, {rtype: [(subject, object)]})
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1379
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1380
    def glob_add_relations(self, session, relations):
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1381
        """add several relations to the repository
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1382
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1383
        relations is a dictionary rtype: [(subj_eid, obj_eid), ...]
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1384
        """
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1385
        sources = {}
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1386
        for rtype, eids_subj_obj in relations.iteritems():
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1387
            if server.DEBUG & server.DBG_REPO:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1388
                for subject, object in relations:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1389
                    print 'ADD relation', subject, rtype, object
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1390
            for subject, object in eids_subj_obj:
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1391
                source = self.locate_relation_source(session, subject, rtype, object)
7238
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1392
                if source not in sources:
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1393
                    relations_by_rtype = {}
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1394
                    sources[source] = relations_by_rtype
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1395
                else:
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1396
                    relations_by_rtype = sources[source]
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1397
                if rtype in relations_by_rtype:
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1398
                    relations_by_rtype[rtype].append((subject, object))
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1399
                else:
7238
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1400
                    relations_by_rtype[rtype] = [(subject, object)]
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1401
        for source, relations_by_rtype in sources.iteritems():
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1402
            if source.should_call_hooks:
7238
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1403
                for rtype, source_relations in relations_by_rtype.iteritems():
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1404
                    for subject, object in source_relations:
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1405
                        del_existing_rel_if_needed(session, subject, rtype, object)
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1406
                    self.hm.call_hooks('before_add_relation', session,
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1407
                                    rtype=rtype, eids_from_to=source_relations)
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1408
            for rtype, source_relations in relations_by_rtype.iteritems():
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1409
                source.add_relations(session, rtype, source_relations)
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1410
                rschema = self.schema.rschema(rtype)
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1411
                for subject, object in source_relations:
7238
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1412
                    session.update_rel_cache_add(subject, rtype, object, rschema.symmetric)
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1413
            if source.should_call_hooks:
7238
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1414
                for rtype, source_relations in relations_by_rtype.iteritems():
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1415
                    self.hm.call_hooks('after_add_relation', session,
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1416
                                       rtype=rtype, eids_from_to=source_relations)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1417
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1418
    def glob_delete_relation(self, session, subject, rtype, object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1419
        """delete a relation from the repository"""
2600
6cd6c5d11b45 [F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
  1420
        if server.DEBUG & server.DBG_REPO:
6cd6c5d11b45 [F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
  1421
            print 'DELETE relation', subject, rtype, object
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1422
        source = self.locate_relation_source(session, subject, rtype, object)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1423
        if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1424
            self.hm.call_hooks('before_delete_relation', session,
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1425
                               eidfrom=subject, rtype=rtype, eidto=object)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1426
        source.delete_relation(session, subject, rtype, object)
2647
b0a2e779845c enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2641
diff changeset
  1427
        rschema = self.schema.rschema(rtype)
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
  1428
        session.update_rel_cache_del(subject, rtype, object, rschema.symmetric)
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
  1429
        if rschema.symmetric:
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
  1430
            # on symmetric relation, we can't now in which sense it's
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1431
            # stored so try to delete both
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1432
            source.delete_relation(session, object, rtype, subject)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1433
        if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1434
            self.hm.call_hooks('after_delete_relation', session,
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1435
                               eidfrom=subject, rtype=rtype, eidto=object)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1436
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1437
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1438
    # pyro handling ###########################################################
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1439
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1440
    @property
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1441
    @cached
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1442
    def pyro_appid(self):
6061
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1443
        from logilab.common import pyro_ext as pyro
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1444
        config = self.config
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1445
        appid = '%s.%s' % pyro.ns_group_and_id(
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1446
            config['pyro-instance-id'] or config.appid,
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1447
            config['pyro-ns-group'])
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1448
        # ensure config['pyro-instance-id'] is a full qualified pyro name
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1449
        config['pyro-instance-id'] = appid
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1450
        return appid
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1451
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1452
    def _use_pyrons(self):
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1453
        """return True if the pyro-ns-host is set to something else
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1454
        than NO_PYRONS, meaning we want to go through a pyro
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1455
        nameserver"""
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1456
        return self.config['pyro-ns-host'] != 'NO_PYRONS'
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1457
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1458
    def pyro_register(self, host=''):
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1459
        """register the repository as a pyro object"""
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1460
        from logilab.common import pyro_ext as pyro
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1461
        daemon = pyro.register_object(self, self.pyro_appid,
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1462
                                      daemonhost=self.config['pyro-host'],
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1463
                                      nshost=self.config['pyro-ns-host'],
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1464
                                      use_pyrons=self._use_pyrons())
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1465
        self.info('repository registered as a pyro object %s', self.pyro_appid)
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1466
        self.pyro_uri =  pyro.get_object_uri(self.pyro_appid)
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1467
        self.info('pyro uri is: %s', self.pyro_uri)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1468
        self.pyro_registered = True
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1469
        # register a looping task to regularly ensure we're still registered
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1470
        # into the pyro name server
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1471
        if self._use_pyrons():
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1472
            self.looping_task(60*10, self._ensure_pyro_ns)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1473
        return daemon
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1474
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1475
    def _ensure_pyro_ns(self):
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1476
        if not self._use_pyrons():
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1477
            return
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1478
        from logilab.common import pyro_ext as pyro
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1479
        pyro.ns_reregister(self.pyro_appid, nshost=self.config['pyro-ns-host'])
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1480
        self.info('repository re-registered as a pyro object %s',
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1481
                  self.pyro_appid)
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1482
1228
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1483
    # multi-sources planner helpers ###########################################
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1484
1228
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1485
    @cached
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1486
    def rel_type_sources(self, rtype):
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1487
        return tuple([source for source in self.sources
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1488
                      if source.support_relation(rtype)
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1489
                      or rtype in source.dont_cross_relations])
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1490
1228
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1491
    @cached
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1492
    def can_cross_relation(self, rtype):
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1493
        return tuple([source for source in self.sources
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1494
                      if source.support_relation(rtype)
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1495
                      and rtype in source.cross_relations])
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1496
1228
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1497
    @cached
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1498
    def is_multi_sources_relation(self, rtype):
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1499
        return any(source for source in self.sources
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1500
                   if not source is self.system_source
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1501
                   and source.support_relation(rtype))
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1502
7083
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7064
diff changeset
  1503
    # these are overridden by set_log_methods below
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7064
diff changeset
  1504
    # only defining here to prevent pylint from complaining
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7064
diff changeset
  1505
    info = warning = error = critical = exception = debug = lambda msg,*a,**kw: None
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1506
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1507
def pyro_unregister(config):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1508
    """unregister the repository from the pyro name server"""
2665
0c6281487f90 [pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2650
diff changeset
  1509
    from logilab.common.pyro_ext import ns_unregister
3539
f3b14d052798 [pyro] merge pyro-id / pyro-instance-id options, put all pyro options in the same section of the configuration file
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3250
diff changeset
  1510
    appid = config['pyro-instance-id'] or config.appid
2665
0c6281487f90 [pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2650
diff changeset
  1511
    ns_unregister(appid, config['pyro-ns-group'], config['pyro-ns-host'])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1512
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1513
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1514
from logging import getLogger
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1515
from cubicweb import set_log_methods
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1516
set_log_methods(Repository, getLogger('cubicweb.repository'))