server/repository.py
author Aurelien Campeas <aurelien.campeas@logilab.fr>
Mon, 13 Jan 2014 15:56:52 +0100
changeset 9404 3e3e9b37e177
parent 9402 2c48c091b6a2
child 9448 3e7cad3967c5
permissions -rw-r--r--
fix version number: we still target cw 3.19
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
8943
58b3b2d9c965 [repo] kill rebuildinfered feature from Repository.set_schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8916
diff changeset
     1
# copyright 2003-2013 LOGILAB S.A. (Paris, FRANCE), all rights reserved.
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     2
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     3
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     4
# This file is part of CubicWeb.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     5
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     6
# CubicWeb is free software: you can redistribute it and/or modify it under the
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     7
# terms of the GNU Lesser General Public License as published by the Free
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     8
# Software Foundation, either version 2.1 of the License, or (at your option)
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
     9
# any later version.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    10
#
5424
8ecbcbff9777 replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5421
diff changeset
    11
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT
5421
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    12
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    13
# FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public License for more
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    14
# details.
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    15
#
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    16
# You should have received a copy of the GNU Lesser General Public License along
8167de96c523 proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5376
diff changeset
    17
# with CubicWeb.  If not, see <http://www.gnu.org/licenses/>.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    18
"""Defines the central class for the CubicWeb RQL server: the repository.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    19
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    20
The repository is an abstraction allowing execution of rql queries against
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    21
data sources. Most of the work is actually done in helper classes. The
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    22
repository mainly:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    23
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    24
* brings these classes all together to provide a single access
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2473
diff changeset
    25
  point to a cubicweb instance.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    26
* handles session management
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    27
* provides method for pyro registration, to call if pyro is enabled
5587
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
    28
"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    29
__docformat__ = "restructuredtext en"
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    30
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    31
import sys
5587
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
    32
import threading
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    33
import Queue
7507
4c043afb104a fix failures introduced by recent refactoring
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7503
diff changeset
    34
from warnings import warn
6390
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
    35
from itertools import chain
4719
aaed3f813ef8 kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4714
diff changeset
    36
from os.path import join
1016
26387b836099 use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents: 636
diff changeset
    37
from datetime import datetime
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    38
from time import time, localtime, strftime
9113
af6efc15fc90 [repository] add an ``internal_cnx`` method to replace ``internal_session``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9105
diff changeset
    39
from contextlib import contextmanager
9302
af36f459d2f0 [server] deprecate the old multi-source system
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9272
diff changeset
    40
from warnings import warn
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    41
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
    42
from logilab.common.decorators import cached, clear_cache
9034
cc3442054e48 [repo] move repo stats to Service (closes #2951067)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9033
diff changeset
    43
from logilab.common.deprecation import deprecated
3757
122a01751d59 2.4 compat
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3694
diff changeset
    44
from logilab.common.compat import any
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
    45
from logilab.common import flatten
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    46
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    47
from yams import BadSchemaDefinition
5030
5238d9a8dfee [form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4984
diff changeset
    48
from yams.schema import role_name
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    49
from rql import RQLSyntaxError
6390
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
    50
from rql.utils import rqlvar_maker
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    51
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
    52
from cubicweb import (CW_SOFTWARE_ROOT, CW_MIGRATION_MAP, QueryError,
2709
6ee816eb9f25 [hooksmanager] hooks should be reseted after vreg reload
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2708
diff changeset
    53
                      UnknownEid, AuthenticationError, ExecutionError,
3042
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
    54
                      ETypeNotSupportedBySources, MultiSourcesError,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    55
                      BadConnectionId, Unauthorized, ValidationError,
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8699
diff changeset
    56
                      RepositoryError, UniqueTogetherError, onevent)
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
    57
from cubicweb import cwvreg, schema, server
7573
c8f8762c986d [repo, looping task] raise a custom exception when repository is shuting down, avoid looping task to be restarted in such case. Closes #1021276
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7550
diff changeset
    58
from cubicweb.server import ShuttingDown, utils, hook, pool, querier, sources
8562
0d2fb4604265 [session] fix arguments default value and promote usage of security_enabled as session method. Closes #2481820
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8539
diff changeset
    59
from cubicweb.server.session import Session, InternalSession, InternalManager
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
    60
from cubicweb.server.ssplanner import EditedEntity
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    61
7780
a1d5365fefc1 [cubification] moved CWPermission handling into a localperms cube. closes #1914011
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7723
diff changeset
    62
NO_CACHE_RELATIONS = set( [('owned_by', 'object'),
7373
e5e6ef56cfb5 [repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7305
diff changeset
    63
                           ('created_by', 'object'),
e5e6ef56cfb5 [repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7305
diff changeset
    64
                           ('cw_source', 'object'),
e5e6ef56cfb5 [repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7305
diff changeset
    65
                           ])
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
    66
8456
c912d82f2166 [repository] drop unused argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8433
diff changeset
    67
def prefill_entity_caches(entity):
6467
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    68
    session = entity._cw
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    69
    # prefill entity relation caches
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    70
    for rschema in entity.e_schema.subject_relations():
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    71
        rtype = str(rschema)
7373
e5e6ef56cfb5 [repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7305
diff changeset
    72
        if rtype in schema.VIRTUAL_RTYPES or (rtype, 'subject') in NO_CACHE_RELATIONS:
6467
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    73
            continue
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    74
        if rschema.final:
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    75
            entity.cw_attr_cache.setdefault(rtype, None)
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    76
        else:
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    77
            entity.cw_set_relation_cache(rtype, 'subject',
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    78
                                         session.empty_rset())
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    79
    for rschema in entity.e_schema.object_relations():
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    80
        rtype = str(rschema)
7373
e5e6ef56cfb5 [repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7305
diff changeset
    81
        if rtype in schema.VIRTUAL_RTYPES or (rtype, 'object') in NO_CACHE_RELATIONS:
6467
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    82
            continue
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    83
        entity.cw_set_relation_cache(rtype, 'object', session.empty_rset())
869813c2dd40 [entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6466
diff changeset
    84
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    85
def del_existing_rel_if_needed(session, eidfrom, rtype, eidto):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    86
    """delete existing relation when adding a new one if card is 1 or ?
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    87
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    88
    have to be done once the new relation has been inserted to avoid having
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    89
    an entity without a relation for some time
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    90
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    91
    this kind of behaviour has to be done in the repository so we don't have
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    92
    hooks order hazardness
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    93
    """
7503
bc30c2faaadc [repository] refactor and optimize '?1' relation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7502
diff changeset
    94
    # skip that if integrity explicitly disabled
bc30c2faaadc [repository] refactor and optimize '?1' relation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7502
diff changeset
    95
    if not session.is_hook_category_activated('activeintegrity'):
4687
082e66184f71 [integrity] skip auto-delete for internal sessions (but *not* for super session, see 4642:921737d2e3a8)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4686
diff changeset
    96
        return
7502
e7190f7e850e [session] deprecates schema_rproperty in favor of more optimized rtype_eids_rdef which return the rdef (so reusable to gather other data)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
    97
    rdef = session.rtype_eids_rdef(rtype, eidfrom, eidto)
e7190f7e850e [session] deprecates schema_rproperty in favor of more optimized rtype_eids_rdef which return the rdef (so reusable to gather other data)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7501
diff changeset
    98
    card = rdef.cardinality
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
    99
    # one may be tented to check for neweids but this may cause more than one
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   100
    # relation even with '1?'  cardinality if thoses relations are added in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   101
    # same transaction where the entity is being created. This never occurs from
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   102
    # the web interface but may occurs during test or dbapi connection (though
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   103
    # not expected for this).  So: don't do it, we pretend to ensure repository
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   104
    # consistency.
4673
6f8b925a29f4 auto-delete relation with single cardinality using execute, not unsafe_execute, so permissions are correctly checked
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4643
diff changeset
   105
    #
6189
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   106
    # notes:
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   107
    # * inlined relations will be implicitly deleted for the subject entity
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   108
    # * we don't want read permissions to be applied but we want delete
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   109
    #   permission to be checked
7503
bc30c2faaadc [repository] refactor and optimize '?1' relation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7502
diff changeset
   110
    if card[0] in '1?':
8562
0d2fb4604265 [session] fix arguments default value and promote usage of security_enabled as session method. Closes #2481820
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8539
diff changeset
   111
        with session.security_enabled(read=False):
6189
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   112
            session.execute('DELETE X %s Y WHERE X eid %%(x)s, '
9fa02f352c8c cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6188
diff changeset
   113
                            'NOT Y eid %%(y)s' % rtype,
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
   114
                                {'x': eidfrom, 'y': eidto})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   115
    if card[1] in '1?':
8562
0d2fb4604265 [session] fix arguments default value and promote usage of security_enabled as session method. Closes #2481820
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8539
diff changeset
   116
        with session.security_enabled(read=False):
5711
ea2538a3c622 [repo] fix typo
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5641
diff changeset
   117
            session.execute('DELETE X %s Y WHERE Y eid %%(y)s, '
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
   118
                            'NOT X eid %%(x)s' % rtype,
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
   119
                            {'x': eidfrom, 'y': eidto})
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   120
3694
33dbb1da1db9 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3689
diff changeset
   121
8547
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   122
def preprocess_inlined_relations(session, entity):
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   123
    """when an entity is added, check if it has some inlined relation which
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   124
    requires to be extrated for proper call hooks
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   125
    """
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   126
    relations = []
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   127
    activeintegrity = session.is_hook_category_activated('activeintegrity')
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   128
    eschema = entity.e_schema
8696
0bb18407c053 [toward py3k] rewrite dict.keys() and dict.values() (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8695
diff changeset
   129
    for attr in entity.cw_edited:
8547
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   130
        rschema = eschema.subjrels[attr]
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   131
        if not rschema.final: # inlined relation
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   132
            value = entity.cw_edited[attr]
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   133
            relations.append((attr, value))
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   134
            session.update_rel_cache_add(entity.eid, attr, value)
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   135
            rdef = session.rtype_eids_rdef(attr, entity.eid, value)
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   136
            if rdef.cardinality[1] in '1?' and activeintegrity:
8573
ae0a567dff30 backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8562 8548
diff changeset
   137
                with session.security_enabled(read=False):
8547
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   138
                    session.execute('DELETE X %s Y WHERE Y eid %%(y)s' % attr,
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   139
                                    {'x': entity.eid, 'y': value})
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   140
    return relations
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   141
8320
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8280
diff changeset
   142
8211
543e1579ba0d [repo] Add a publish/subscribe mechanism for inter-instance communication using zmq
Julien Cristau <julien.cristau@logilab.fr>, Quentin Roquefort <quentin@kpsule.me>
parents: 8190
diff changeset
   143
class NullEventBus(object):
8214
ce9556358dbd [event bus] event has been renamed to publish
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8211
diff changeset
   144
    def publish(self, msg):
8211
543e1579ba0d [repo] Add a publish/subscribe mechanism for inter-instance communication using zmq
Julien Cristau <julien.cristau@logilab.fr>, Quentin Roquefort <quentin@kpsule.me>
parents: 8190
diff changeset
   145
        pass
543e1579ba0d [repo] Add a publish/subscribe mechanism for inter-instance communication using zmq
Julien Cristau <julien.cristau@logilab.fr>, Quentin Roquefort <quentin@kpsule.me>
parents: 8190
diff changeset
   146
543e1579ba0d [repo] Add a publish/subscribe mechanism for inter-instance communication using zmq
Julien Cristau <julien.cristau@logilab.fr>, Quentin Roquefort <quentin@kpsule.me>
parents: 8190
diff changeset
   147
    def add_subscription(self, topic, callback):
543e1579ba0d [repo] Add a publish/subscribe mechanism for inter-instance communication using zmq
Julien Cristau <julien.cristau@logilab.fr>, Quentin Roquefort <quentin@kpsule.me>
parents: 8190
diff changeset
   148
        pass
543e1579ba0d [repo] Add a publish/subscribe mechanism for inter-instance communication using zmq
Julien Cristau <julien.cristau@logilab.fr>, Quentin Roquefort <quentin@kpsule.me>
parents: 8190
diff changeset
   149
543e1579ba0d [repo] Add a publish/subscribe mechanism for inter-instance communication using zmq
Julien Cristau <julien.cristau@logilab.fr>, Quentin Roquefort <quentin@kpsule.me>
parents: 8190
diff changeset
   150
    def start(self):
543e1579ba0d [repo] Add a publish/subscribe mechanism for inter-instance communication using zmq
Julien Cristau <julien.cristau@logilab.fr>, Quentin Roquefort <quentin@kpsule.me>
parents: 8190
diff changeset
   151
        pass
543e1579ba0d [repo] Add a publish/subscribe mechanism for inter-instance communication using zmq
Julien Cristau <julien.cristau@logilab.fr>, Quentin Roquefort <quentin@kpsule.me>
parents: 8190
diff changeset
   152
543e1579ba0d [repo] Add a publish/subscribe mechanism for inter-instance communication using zmq
Julien Cristau <julien.cristau@logilab.fr>, Quentin Roquefort <quentin@kpsule.me>
parents: 8190
diff changeset
   153
    def stop(self):
543e1579ba0d [repo] Add a publish/subscribe mechanism for inter-instance communication using zmq
Julien Cristau <julien.cristau@logilab.fr>, Quentin Roquefort <quentin@kpsule.me>
parents: 8190
diff changeset
   154
        pass
543e1579ba0d [repo] Add a publish/subscribe mechanism for inter-instance communication using zmq
Julien Cristau <julien.cristau@logilab.fr>, Quentin Roquefort <quentin@kpsule.me>
parents: 8190
diff changeset
   155
543e1579ba0d [repo] Add a publish/subscribe mechanism for inter-instance communication using zmq
Julien Cristau <julien.cristau@logilab.fr>, Quentin Roquefort <quentin@kpsule.me>
parents: 8190
diff changeset
   156
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   157
class Repository(object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   158
    """a repository provides access to a set of persistent storages for
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   159
    entities and relations
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   160
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   161
    XXX protect pyro access
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   162
    """
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   163
8322
cb838b126b07 [repository] move task manager instantiation outside repository.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8321
diff changeset
   164
    def __init__(self, config, tasks_manager=None, vreg=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   165
        self.config = config
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   166
        if vreg is None:
8190
2a3c1b787688 [vreg] move base registry implementation to logilab.common. Closes #1916014
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8133
diff changeset
   167
            vreg = cwvreg.CWRegistryStore(config)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   168
        self.vreg = vreg
8322
cb838b126b07 [repository] move task manager instantiation outside repository.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8321
diff changeset
   169
        self._tasks_manager = tasks_manager
8320
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8280
diff changeset
   170
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   171
        self.pyro_registered = False
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
   172
        self.pyro_uri = None
9007
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
   173
        # every pyro client is handled in its own thread; map these threads to
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
   174
        # the session we opened for them so we can clean up when they go away
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
   175
        self._pyro_sessions = {}
8211
543e1579ba0d [repo] Add a publish/subscribe mechanism for inter-instance communication using zmq
Julien Cristau <julien.cristau@logilab.fr>, Quentin Roquefort <quentin@kpsule.me>
parents: 8190
diff changeset
   176
        self.app_instances_bus = NullEventBus()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   177
        self.info('starting repository from %s', self.config.apphome)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   178
        # dictionary of opened sessions
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   179
        self._sessions = {}
8320
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8280
diff changeset
   180
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8280
diff changeset
   181
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   182
        # list of functions to be called at regular interval
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   183
        # list of running threads
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   184
        self._running_threads = []
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   185
        # initial schema, should be build or replaced latter
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   186
        self.schema = schema.CubicWebSchema(config.appid)
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3042
diff changeset
   187
        self.vreg.schema = self.schema # until actual schema is loaded...
5906
d40ced753291 [repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5813
diff changeset
   188
        # shutdown flag
d40ced753291 [repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5813
diff changeset
   189
        self.shutting_down = False
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   190
        # sources (additional sources info in the system database)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   191
        self.system_source = self.get_source('native', 'system',
8672
e8afecc3b576 [repository sources] copy source configuration: source's __init__ modify it while config.sources() return a cached dictionary which shouldn't be modified
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8671
diff changeset
   192
                                             config.sources()['system'].copy())
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   193
        self.sources = [self.system_source]
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   194
        self.sources_by_uri = {'system': self.system_source}
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   195
        # querier helper, need to be created after sources initialization
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   196
        self.querier = querier.QuerierHelper(self, self.schema)
7543
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
   197
        # cache eid -> (type, physical source, extid, actual source)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   198
        self._type_source_cache = {}
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   199
        # cache (extid, source uri) -> eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   200
        self._extid_cache = {}
8946
fae2f561b0f5 [repo] minor cleanups to bootstrap sequence
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8943
diff changeset
   201
        # open some connection sets
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   202
        if config.init_cnxset_pool:
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   203
            self.init_cnxset_pool()
8946
fae2f561b0f5 [repo] minor cleanups to bootstrap sequence
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8943
diff changeset
   204
        # the hooks manager
fae2f561b0f5 [repo] minor cleanups to bootstrap sequence
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8943
diff changeset
   205
        self.hm = hook.HooksManager(self.vreg)
fae2f561b0f5 [repo] minor cleanups to bootstrap sequence
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8943
diff changeset
   206
        # registry hook to fix user class on registry reload
5079
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   207
        @onevent('after-registry-reload', self)
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   208
        def fix_user_classes(self):
8277
8b173e0a617f {repository] add documentation explaining ``fix_user_classes`` hook
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8275
diff changeset
   209
            # After registery reload the 'CWUser' class used for CWEtype
8b173e0a617f {repository] add documentation explaining ``fix_user_classes`` hook
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8275
diff changeset
   210
            # changed.  To any existing user object have a different class than
8b173e0a617f {repository] add documentation explaining ``fix_user_classes`` hook
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8275
diff changeset
   211
            # the new loaded one. We are hot fixing this.
5079
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   212
            usercls = self.vreg['etypes'].etype_class('CWUser')
8696
0bb18407c053 [toward py3k] rewrite dict.keys() and dict.values() (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8695
diff changeset
   213
            for session in self._sessions.itervalues():
5079
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   214
                if not isinstance(session.user, InternalManager):
e646047f80cb [reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5073
diff changeset
   215
                    session.user.__class__ = usercls
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2929
diff changeset
   216
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   217
    def init_cnxset_pool(self):
8946
fae2f561b0f5 [repo] minor cleanups to bootstrap sequence
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8943
diff changeset
   218
        """should be called bootstrap_repository, as this is what it does"""
2959
daabb9bc5233 make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2929
diff changeset
   219
        config = self.config
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   220
        self._cnxsets_pool = Queue.Queue()
8947
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   221
        # 0. init a cnxset that will be used to fetch bootstrap information from
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   222
        #    the database
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   223
        self._cnxsets_pool.put_nowait(pool.ConnectionsSet(self.sources))
8947
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   224
        # 1. set used cubes
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   225
        if config.creating or not config.read_instance_schema:
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   226
            config.bootstrap_cubes()
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   227
        else:
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   228
            self.set_schema(self.config.load_bootstrap_schema(), resetvreg=False)
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   229
            config.init_cubes(self.get_cubes())
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   230
        # 2. load schema
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   231
        if config.quick_start:
8946
fae2f561b0f5 [repo] minor cleanups to bootstrap sequence
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8943
diff changeset
   232
            # quick start: only to get a minimal repository to get cubes
5084
d8f491cb046c [repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5079
diff changeset
   233
            # information (eg dump/restore/...)
8947
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   234
            #
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   235
            # restrict appobject_path to only load hooks and entity classes in
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   236
            # the registry
8537
e30d0a7f0087 [config] turn internal configuration methods building appobjects search path into normal method rather than class method
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   237
            config.cube_appobject_path = set(('hooks', 'entities'))
e30d0a7f0087 [config] turn internal configuration methods building appobjects search path into normal method rather than class method
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
   238
            config.cubicweb_appobject_path = set(('hooks', 'entities'))
8947
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   239
            # limit connections pool to 1
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   240
            config['connections-pool-size'] = 1
8947
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   241
        if config.quick_start or config.creating or not config.read_instance_schema:
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   242
            # load schema from the file system
8580
d753d6a6798f [repository] move modification of appobject_path to repository initialization code so we can restore it later to avoid side effect on the config. Fix regression introduced in d32ab8570e5d
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8573
diff changeset
   243
            if not config.creating:
d753d6a6798f [repository] move modification of appobject_path to repository initialization code so we can restore it later to avoid side effect on the config. Fix regression introduced in d32ab8570e5d
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8573
diff changeset
   244
                self.warning("set fs instance'schema")
9165
7ebf3b1cdb4c Use the list of cubes from the filesystem when reading the schema from the filesystem
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9132
diff changeset
   245
            self.set_schema(config.load_schema(expand_cubes=True))
8592
df16bd045cae [server] fix repository initialisation
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8580
diff changeset
   246
        else:
df16bd045cae [server] fix repository initialisation
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8580
diff changeset
   247
            # normal start: load the instance schema from the database
8947
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   248
            self.info('loading schema from the repository')
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   249
            self.set_schema(self.deserialize_schema())
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   250
        # 3. initialize data sources
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   251
        if config.creating:
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
   252
            # call init_creating so that for instance native source can
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
   253
            # configurate tsearch according to postgres version
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   254
            for source in self.sources:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   255
                source.init_creating()
8947
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   256
        else:
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   257
            self.init_sources_from_database()
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   258
            if 'CWProperty' in self.schema:
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   259
                self.vreg.init_properties(self.properties())
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   260
        # 4. close initialization connection set and reopen fresh ones for
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   261
        #    proper initialization
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   262
        self._get_cnxset().close(True)
8946
fae2f561b0f5 [repo] minor cleanups to bootstrap sequence
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8943
diff changeset
   263
        self.cnxsets = [] # list of available cnxsets (can't iterate on a Queue)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   264
        for i in xrange(config['connections-pool-size']):
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   265
            self.cnxsets.append(pool.ConnectionsSet(self.sources))
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   266
            self._cnxsets_pool.put_nowait(self.cnxsets[-1])
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   267
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   268
    # internals ###############################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   269
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   270
    def init_sources_from_database(self):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   271
        self.sources_by_eid = {}
6664
7b6acf314a7a [repo] don't attempt to read sources from database on quick start. Fix crash after restoring a pre 3.10 database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6629
diff changeset
   272
        if self.config.quick_start \
7b6acf314a7a [repo] don't attempt to read sources from database on quick start. Fix crash after restoring a pre 3.10 database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6629
diff changeset
   273
               or not 'CWSource' in self.schema: # # 3.10 migration
7550
183a61d1bab9 [3.10] fix typo and error for pre 3.10 migration w/ postgres: we *must* call init_creating else the fti isn't correctly initialized, which may lead to silently rollbacked transaction
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7513
diff changeset
   274
            self.system_source.init_creating()
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   275
            return
8433
ff9d6d269877 [server/session,repo] turn InternalSession, hence repo.internal_session, into a context manager (closes #2393651)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8393
diff changeset
   276
        with self.internal_session() as session:
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   277
            # FIXME: sources should be ordered (add_entity priority)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   278
            for sourceent in session.execute(
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   279
                'Any S, SN, SA, SC WHERE S is_instance_of CWSource, '
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   280
                'S name SN, S type SA, S config SC').entities():
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   281
                if sourceent.name == 'system':
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   282
                    self.system_source.eid = sourceent.eid
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   283
                    self.sources_by_eid[sourceent.eid] = self.system_source
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   284
                    self.system_source.init(True, sourceent)
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   285
                    continue
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   286
                self.add_source(sourceent, add_to_cnxsets=False)
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   287
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   288
    def _clear_planning_caches(self):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   289
        for cache in ('source_defs', 'is_multi_sources_relation',
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   290
                      'can_cross_relation', 'rel_type_sources'):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   291
            clear_cache(self, cache)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   292
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   293
    def add_source(self, sourceent, add_to_cnxsets=True):
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   294
        source = self.get_source(sourceent.type, sourceent.name,
6945
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6941
diff changeset
   295
                                 sourceent.host_config, sourceent.eid)
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   296
        self.sources_by_eid[sourceent.eid] = source
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   297
        self.sources_by_uri[sourceent.name] = source
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   298
        if self.config.source_enabled(source):
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   299
            # call source's init method to complete their initialisation if
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   300
            # needed (for instance looking for persistent configuration using an
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   301
            # internal session, which is not possible until connections sets have been
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   302
            # initialized)
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   303
            source.init(True, sourceent)
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   304
            if not source.copy_based_source:
9302
af36f459d2f0 [server] deprecate the old multi-source system
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9272
diff changeset
   305
                warn('[3.18] old multi-source system will go away in the next version',
af36f459d2f0 [server] deprecate the old multi-source system
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9272
diff changeset
   306
                     DeprecationWarning)
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   307
                self.sources.append(source)
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   308
                self.querier.set_planner()
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   309
                if add_to_cnxsets:
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   310
                    for cnxset in self.cnxsets:
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   311
                       cnxset.add_source(source)
6724
24bf6f181d0e [pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
   312
        else:
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   313
            source.init(False, sourceent)
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   314
        self._clear_planning_caches()
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   315
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   316
    def remove_source(self, uri):
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   317
        source = self.sources_by_uri.pop(uri)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   318
        del self.sources_by_eid[source.eid]
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
   319
        if self.config.source_enabled(source) and not source.copy_based_source:
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   320
            self.sources.remove(source)
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   321
            self.querier.set_planner()
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   322
            for cnxset in self.cnxsets:
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   323
                cnxset.remove_source(source)
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   324
        self._clear_planning_caches()
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   325
6945
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6941
diff changeset
   326
    def get_source(self, type, uri, source_config, eid=None):
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   327
        # set uri and type in source config so it's available through
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   328
        # source_defs()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   329
        source_config['uri'] = uri
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   330
        source_config['type'] = type
6945
28bf94d062a9 [sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6941
diff changeset
   331
        return sources.get_source(type, source_config, self, eid)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   332
8943
58b3b2d9c965 [repo] kill rebuildinfered feature from Repository.set_schema
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8916
diff changeset
   333
    def set_schema(self, schema, resetvreg=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   334
        self.info('set schema %s %#x', schema.name, id(schema))
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3042
diff changeset
   335
        if resetvreg:
5447
f29dafba250a [vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5442
diff changeset
   336
            # trigger full reload of all appobjects
3240
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3042
diff changeset
   337
            self.vreg.set_schema(schema)
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3042
diff changeset
   338
        else:
8604a15995d1 refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3042
diff changeset
   339
            self.vreg._set_schema(schema)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   340
        self.querier.set_schema(schema)
6126
aca6a2c357fd [repository] enabled sources refactoring: to avoid error, we should always have all known source in repo.sources_by_uri and only enabled ones in repo.sources, so we still have access to the definition of temporarily disabled sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6061
diff changeset
   341
        # don't use self.sources, we may want to give schema even to disabled
aca6a2c357fd [repository] enabled sources refactoring: to avoid error, we should always have all known source in repo.sources_by_uri and only enabled ones in repo.sources, so we still have access to the definition of temporarily disabled sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6061
diff changeset
   342
        # sources
8696
0bb18407c053 [toward py3k] rewrite dict.keys() and dict.values() (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8695
diff changeset
   343
        for source in self.sources_by_uri.itervalues():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   344
            source.set_schema(schema)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   345
        self.schema = schema
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   346
8947
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   347
    def deserialize_schema(self):
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   348
        """load schema from the database"""
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   349
        from cubicweb.server.schemaserial import deserialize_schema
2839
6419af16faa0 imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2835
diff changeset
   350
        appschema = schema.CubicWebSchema(self.config.appid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   351
        self.debug('deserializing db schema into %s %#x', appschema.name, id(appschema))
8433
ff9d6d269877 [server/session,repo] turn InternalSession, hence repo.internal_session, into a context manager (closes #2393651)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8393
diff changeset
   352
        with self.internal_session() as session:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   353
            try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   354
                deserialize_schema(appschema, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   355
            except BadSchemaDefinition:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   356
                raise
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8694
diff changeset
   357
            except Exception as ex:
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   358
                import traceback
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   359
                traceback.print_exc()
9171
be9596750678 Fix two crashes in db-check (closes #3024964)
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9165
diff changeset
   360
                raise (Exception('Is the database initialised ? (cause: %s)' % ex),
be9596750678 Fix two crashes in db-check (closes #3024964)
Denis Laxalde <denis.laxalde@logilab.fr>
parents: 9165
diff changeset
   361
                       None, sys.exc_info()[-1])
8947
3bbd416b09ec [repo] straightforward bootstrap sequence. Closes #2841188
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8946
diff changeset
   362
        return appschema
8321
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   363
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   364
    def _prepare_startup(self):
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   365
        """Prepare "Repository as a server" for startup.
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   366
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   367
        * trigger server startup hook,
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   368
        * register session clean up task.
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   369
        """
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   370
        if not (self.config.creating or self.config.repairing
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   371
                or self.config.quick_start):
4958
665eacdd8c50 [repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   372
            # call instance level initialisation hooks
665eacdd8c50 [repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   373
            self.hm.call_hooks('server_startup', repo=self)
665eacdd8c50 [repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4951
diff changeset
   374
            # register a task to cleanup expired session
6012
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   375
            self.cleanup_session_time = self.config['cleanup-session-time'] or 60 * 60 * 24
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   376
            assert self.cleanup_session_time > 0
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   377
            cleanup_session_interval = min(60*60, self.cleanup_session_time / 3)
8322
cb838b126b07 [repository] move task manager instantiation outside repository.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8321
diff changeset
   378
            assert self._tasks_manager is not None, "This Repository is not intended to be used as a server"
8321
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   379
            self._tasks_manager.add_looping_task(cleanup_session_interval,
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   380
                                                 self.clean_sessions)
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   381
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   382
    def start_looping_tasks(self):
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   383
        """Actual "Repository as a server" startup.
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   384
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   385
        * trigger server startup hook,
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   386
        * register session clean up task,
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   387
        * start all tasks.
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   388
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   389
        XXX Other startup related stuffs are done elsewhere. In Repository
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   390
        XXX __init__ or in external codes (various server managers).
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   391
        """
b5d5a5630649 [repository] split repo initialization from starting looping task (closes #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8320
diff changeset
   392
        self._prepare_startup()
8322
cb838b126b07 [repository] move task manager instantiation outside repository.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8321
diff changeset
   393
        assert self._tasks_manager is not None, "This Repository is not intended to be used as a server"
8320
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8280
diff changeset
   394
        self._tasks_manager.start()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   395
2708
60d728bdcba5 allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2667
diff changeset
   396
    def looping_task(self, interval, func, *args):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   397
        """register a function to be called every `interval` seconds.
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   398
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   399
        looping tasks can only be registered during repository initialization,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   400
        once done this method will fail.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   401
        """
8322
cb838b126b07 [repository] move task manager instantiation outside repository.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8321
diff changeset
   402
        assert self._tasks_manager is not None, "This Repository is not intended to be used as a server"
8320
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8280
diff changeset
   403
        self._tasks_manager.add_looping_task(interval, func, *args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   404
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   405
    def threaded_task(self, func):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   406
        """start function in a separated thread"""
8320
cd2d332b3063 [repo looping task] move looping task logic in a dedicated object (progress #2204047)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8280
diff changeset
   407
        utils.RepoThread(func, self._running_threads).start()
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   408
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   409
    #@locked
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   410
    def _get_cnxset(self):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   411
        try:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   412
            return self._cnxsets_pool.get(True, timeout=5)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   413
        except Queue.Empty:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   414
            raise Exception('no connections set available after 5 secs, probably either a '
9267
24d9b86dfa54 spelling: rollbacked -> rolled back
Rémi Cardona <remi.cardona@logilab.fr>
parents: 9226
diff changeset
   415
                            'bug in code (too many uncommited/rolled back '
4706
6035e96b64dd added stats for munin collecting #615844 - from 027bbff3659f
arthur
parents: 4689
diff changeset
   416
                            'connections) or too much load on the server (in '
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   417
                            'which case you can try to set a bigger '
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   418
                            'connections pool size)')
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   419
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   420
    def _free_cnxset(self, cnxset):
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   421
        self._cnxsets_pool.put_nowait(cnxset)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   422
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   423
    def pinfo(self):
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   424
        # XXX: session.cnxset is accessed from a local storage, would be interesting
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   425
        #      to see if there is a cnxset set in any thread specific data)
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   426
        return '%s: %s (%s)' % (self._cnxsets_pool.qsize(),
8696
0bb18407c053 [toward py3k] rewrite dict.keys() and dict.values() (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8695
diff changeset
   427
                                ','.join(session.user.login for session in self._sessions.itervalues()
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   428
                                         if session.cnxset),
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   429
                                threading.currentThread())
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   430
    def shutdown(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   431
        """called on server stop event to properly close opened sessions and
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   432
        connections
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   433
        """
5906
d40ced753291 [repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5813
diff changeset
   434
        assert not self.shutting_down, 'already shutting down'
8393
77c7158916c1 [events] add a new before_shutdown event (closes #2345728)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8366
diff changeset
   435
        if not (self.config.creating or self.config.repairing
77c7158916c1 [events] add a new before_shutdown event (closes #2345728)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8366
diff changeset
   436
                or self.config.quick_start):
77c7158916c1 [events] add a new before_shutdown event (closes #2345728)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8366
diff changeset
   437
            # then, the system source is still available
77c7158916c1 [events] add a new before_shutdown event (closes #2345728)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8366
diff changeset
   438
            self.hm.call_hooks('before_server_shutdown', repo=self)
5906
d40ced753291 [repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5813
diff changeset
   439
        self.shutting_down = True
5749
b4393b681f7a [repo] on repository shutdown, we've to close the new eid creation connection
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5747
diff changeset
   440
        self.system_source.shutdown()
8322
cb838b126b07 [repository] move task manager instantiation outside repository.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8321
diff changeset
   441
        if self._tasks_manager is not None:
cb838b126b07 [repository] move task manager instantiation outside repository.
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8321
diff changeset
   442
            self._tasks_manager.stop()
8351
02f4f01375e8 [repository] fire 'server_shutdown' hooks before waiting for theads
David Douard <david.douard@logilab.fr>
parents: 8322
diff changeset
   443
        if not (self.config.creating or self.config.repairing
02f4f01375e8 [repository] fire 'server_shutdown' hooks before waiting for theads
David Douard <david.douard@logilab.fr>
parents: 8322
diff changeset
   444
                or self.config.quick_start):
02f4f01375e8 [repository] fire 'server_shutdown' hooks before waiting for theads
David Douard <david.douard@logilab.fr>
parents: 8322
diff changeset
   445
            self.hm.call_hooks('server_shutdown', repo=self)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   446
        for thread in self._running_threads:
5376
2c3f14bc2590 [python2.6] don't add a name property on Thread
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5292
diff changeset
   447
            self.info('waiting thread %s...', thread.getName())
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   448
            thread.join()
5376
2c3f14bc2590 [python2.6] don't add a name property on Thread
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5292
diff changeset
   449
            self.info('thread %s finished', thread.getName())
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   450
        self.close_sessions()
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   451
        while not self._cnxsets_pool.empty():
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   452
            cnxset = self._cnxsets_pool.get_nowait()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   453
            try:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   454
                cnxset.close(True)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   455
            except Exception:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   456
                self.exception('error while closing %s' % cnxset)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   457
                continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   458
        if self.pyro_registered:
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
   459
            if self._use_pyrons():
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
   460
                pyro_unregister(self.config)
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
   461
            self.pyro_uri = None
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   462
        hits, misses = self.querier.cache_hit, self.querier.cache_miss
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   463
        try:
5747
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   464
            self.info('rql st cache hit/miss: %s/%s (%s%% hits)', hits, misses,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   465
                      (hits * 100) / (hits + misses))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   466
            hits, misses = self.system_source.cache_hit, self.system_source.cache_miss
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   467
            self.info('sql cache hit/miss: %s/%s (%s%% hits)', hits, misses,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   468
                      (hits * 100) / (hits + misses))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   469
            nocache  = self.system_source.no_cache
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   470
            self.info('sql cache usage: %s/%s (%s%%)', hits+ misses, nocache,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   471
                      ((hits + misses) * 100) / (hits + misses + nocache))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   472
        except ZeroDivisionError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   473
            pass
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   474
6849
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   475
    def check_auth_info(self, session, login, authinfo):
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   476
        """validate authentication, raise AuthenticationError on failure, return
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   477
        associated CWUser's eid on success.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   478
        """
8229
b7bc631816f7 [ldapfeed] make authentication actually working
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8214
diff changeset
   479
        # iter on sources_by_uri then check enabled source since sources doesn't
b7bc631816f7 [ldapfeed] make authentication actually working
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8214
diff changeset
   480
        # contain copy based sources
b7bc631816f7 [ldapfeed] make authentication actually working
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8214
diff changeset
   481
        for source in self.sources_by_uri.itervalues():
b7bc631816f7 [ldapfeed] make authentication actually working
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8214
diff changeset
   482
            if self.config.source_enabled(source) and source.support_entity('CWUser'):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   483
                try:
6849
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   484
                    return source.authenticate(session, login, **authinfo)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   485
                except AuthenticationError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   486
                    continue
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   487
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   488
            raise AuthenticationError('authentication failed with all sources')
6849
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   489
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   490
    def authenticate_user(self, session, login, **authinfo):
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   491
        """validate login / password, raise AuthenticationError on failure
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   492
        return associated CWUser instance on success
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   493
        """
5a0c2cfc19bf [repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6822
diff changeset
   494
        eid = self.check_auth_info(session, login, authinfo)
2268
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   495
        cwuser = self._build_user(session, eid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   496
        if self.config.consider_user_state and \
5556
9ab2b4c74baf [entity] introduce a new 'adapters' registry
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5447
diff changeset
   497
               not cwuser.cw_adapt_to('IWorkflowable').state in cwuser.AUTHENTICABLE_STATES:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   498
            raise AuthenticationError('user is not in authenticable state')
2268
2f336fd5e040 euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2267
diff changeset
   499
        return cwuser
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   500
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   501
    def _build_user(self, session, eid):
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   502
        """return a CWUser entity for user with the given eid"""
9105
55738c9dc26f [session] explicitly take Connection object in close_cnx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9093
diff changeset
   503
        with session.ensure_cnx_set:
55738c9dc26f [session] explicitly take Connection object in close_cnx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9093
diff changeset
   504
            cls = self.vreg['etypes'].etype_class('CWUser')
55738c9dc26f [session] explicitly take Connection object in close_cnx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9093
diff changeset
   505
            st = cls.fetch_rqlst(session.user, ordermethod=None)
55738c9dc26f [session] explicitly take Connection object in close_cnx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9093
diff changeset
   506
            st.add_eid_restriction(st.get_variable('X'), 'x', 'Substitute')
55738c9dc26f [session] explicitly take Connection object in close_cnx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9093
diff changeset
   507
            rset = session.execute(st.as_string(), {'x': eid})
55738c9dc26f [session] explicitly take Connection object in close_cnx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9093
diff changeset
   508
            assert len(rset) == 1, rset
55738c9dc26f [session] explicitly take Connection object in close_cnx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9093
diff changeset
   509
            cwuser = rset.get_entity(0, 0)
55738c9dc26f [session] explicitly take Connection object in close_cnx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9093
diff changeset
   510
            # pylint: disable=W0104
55738c9dc26f [session] explicitly take Connection object in close_cnx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9093
diff changeset
   511
            # prefetch / cache cwuser's groups and properties. This is especially
55738c9dc26f [session] explicitly take Connection object in close_cnx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9093
diff changeset
   512
            # useful for internal sessions to avoid security insertions
55738c9dc26f [session] explicitly take Connection object in close_cnx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9093
diff changeset
   513
            cwuser.groups
55738c9dc26f [session] explicitly take Connection object in close_cnx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9093
diff changeset
   514
            cwuser.properties
55738c9dc26f [session] explicitly take Connection object in close_cnx
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9093
diff changeset
   515
            return cwuser
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   516
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   517
    # public (dbapi) interface ################################################
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   518
9404
3e3e9b37e177 fix version number: we still target cw 3.19
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9402
diff changeset
   519
    @deprecated("[3.19] use _cw.call_service('repo_stats'")
5587
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   520
    def stats(self): # XXX restrict to managers session?
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   521
        """Return a dictionary containing some statistics about the repository
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   522
        resources usage.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   523
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   524
        This is a public method, not requiring a session id.
9034
cc3442054e48 [repo] move repo stats to Service (closes #2951067)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9033
diff changeset
   525
cc3442054e48 [repo] move repo stats to Service (closes #2951067)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9033
diff changeset
   526
        This method is deprecated in favor of using _cw.call_service('repo_stats')
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   527
        """
9034
cc3442054e48 [repo] move repo stats to Service (closes #2951067)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9033
diff changeset
   528
        with self.internal_session() as session:
cc3442054e48 [repo] move repo stats to Service (closes #2951067)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9033
diff changeset
   529
            return session.call_service('repo_stats')
5587
72679e450f6d [web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5426
diff changeset
   530
9404
3e3e9b37e177 fix version number: we still target cw 3.19
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9402
diff changeset
   531
    @deprecated("[3.19] use _cw.call_service('repo_gc_stats'")
8224
e35d4d4f7eb3 [repository] add a ``gc_stats`` method and 2 more cache size for ``stats`` (closes #2179735)
David Douard <david.douard@logilab.fr>
parents: 8133
diff changeset
   532
    def gc_stats(self, nmax=20):
e35d4d4f7eb3 [repository] add a ``gc_stats`` method and 2 more cache size for ``stats`` (closes #2179735)
David Douard <david.douard@logilab.fr>
parents: 8133
diff changeset
   533
        """Return a dictionary containing some statistics about the repository
e35d4d4f7eb3 [repository] add a ``gc_stats`` method and 2 more cache size for ``stats`` (closes #2179735)
David Douard <david.douard@logilab.fr>
parents: 8133
diff changeset
   534
        memory usage.
e35d4d4f7eb3 [repository] add a ``gc_stats`` method and 2 more cache size for ``stats`` (closes #2179735)
David Douard <david.douard@logilab.fr>
parents: 8133
diff changeset
   535
e35d4d4f7eb3 [repository] add a ``gc_stats`` method and 2 more cache size for ``stats`` (closes #2179735)
David Douard <david.douard@logilab.fr>
parents: 8133
diff changeset
   536
        This is a public method, not requiring a session id.
e35d4d4f7eb3 [repository] add a ``gc_stats`` method and 2 more cache size for ``stats`` (closes #2179735)
David Douard <david.douard@logilab.fr>
parents: 8133
diff changeset
   537
e35d4d4f7eb3 [repository] add a ``gc_stats`` method and 2 more cache size for ``stats`` (closes #2179735)
David Douard <david.douard@logilab.fr>
parents: 8133
diff changeset
   538
        nmax is the max number of (most) referenced object returned as
e35d4d4f7eb3 [repository] add a ``gc_stats`` method and 2 more cache size for ``stats`` (closes #2179735)
David Douard <david.douard@logilab.fr>
parents: 8133
diff changeset
   539
        the 'referenced' result
e35d4d4f7eb3 [repository] add a ``gc_stats`` method and 2 more cache size for ``stats`` (closes #2179735)
David Douard <david.douard@logilab.fr>
parents: 8133
diff changeset
   540
        """
9035
63f3d25bab14 [repo] move repo.gc_stats to Service API (closes #2951068)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9034
diff changeset
   541
        with self.internal_session() as session:
63f3d25bab14 [repo] move repo.gc_stats to Service API (closes #2951068)
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9034
diff changeset
   542
            return session.call_service('repo_gc_stats', nmax=nmax)
8224
e35d4d4f7eb3 [repository] add a ``gc_stats`` method and 2 more cache size for ``stats`` (closes #2179735)
David Douard <david.douard@logilab.fr>
parents: 8133
diff changeset
   543
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   544
    def get_schema(self):
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   545
        """Return the instance schema.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   546
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   547
        This is a public method, not requiring a session id.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   548
        """
7723
badfd5524ab6 [repo] Stop setting hashmode on schema
Julien Cristau <julien.cristau@logilab.fr>
parents: 7716
diff changeset
   549
        return self.schema
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   550
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   551
    def get_cubes(self):
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   552
        """Return the list of cubes used by this instance.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   553
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   554
        This is a public method, not requiring a session id.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   555
        """
2473
490f88fb99b6 new distinguish repairing/creating from regular start.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2268
diff changeset
   556
        versions = self.get_versions(not (self.config.creating
4689
4eb1f4490538 [test] skipping versions checking during test is enough, no need for monkey patch
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4687
diff changeset
   557
                                          or self.config.repairing
5043
fe52dd3936cf [repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5032
diff changeset
   558
                                          or self.config.quick_start
4689
4eb1f4490538 [test] skipping versions checking during test is enough, no need for monkey patch
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4687
diff changeset
   559
                                          or self.config.mode == 'test'))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   560
        cubes = list(versions)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   561
        cubes.remove('cubicweb')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   562
        return cubes
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   563
6308
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   564
    def get_option_value(self, option, foreid=None):
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   565
        """Return the value for `option` in the configuration. If `foreid` is
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   566
        specified, the actual repository to which this entity belongs is
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   567
        derefenced and the option value retrieved from it.
6257
7eb5f1aed728 [repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6256
diff changeset
   568
7eb5f1aed728 [repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6256
diff changeset
   569
        This is a public method, not requiring a session id.
7eb5f1aed728 [repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6256
diff changeset
   570
        """
7eb5f1aed728 [repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6256
diff changeset
   571
        # XXX we may want to check we don't give sensible information
8074
a7f76e322659 add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8051
diff changeset
   572
        # XXX the only cube using 'foreid', apycot, stop used this, we probably
a7f76e322659 add some notes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8051
diff changeset
   573
        # want to drop this argument
6308
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   574
        if foreid is None:
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   575
            return self.config[option]
7543
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
   576
        _, sourceuri, extid, _ = self.type_and_source_from_eid(foreid)
6308
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   577
        if sourceuri == 'system':
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   578
            return self.config[option]
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   579
        cnxset = self._get_cnxset()
6308
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   580
        try:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   581
            cnx = cnxset.connection(sourceuri)
7064
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   582
            # needed to check connection is valid and usable by the current
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   583
            # thread
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   584
            newcnx = self.sources_by_uri[sourceuri].check_connection(cnx)
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   585
            if newcnx is not None:
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   586
                cnx = newcnx
b8ce2266d026 [repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6889
diff changeset
   587
            return cnx.get_option_value(option, extid)
6308
c151c730a1ac [dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6257
diff changeset
   588
        finally:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   589
            self._free_cnxset(cnxset)
6257
7eb5f1aed728 [repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6256
diff changeset
   590
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   591
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   592
    def get_versions(self, checkversions=False):
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   593
        """Return the a dictionary containing cubes used by this instance
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   594
        as key with their version as value, including cubicweb version.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   595
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   596
        This is a public method, not requiring a session id.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   597
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   598
        from logilab.common.changelog import Version
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   599
        vcconf = {}
8433
ff9d6d269877 [server/session,repo] turn InternalSession, hence repo.internal_session, into a context manager (closes #2393651)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8393
diff changeset
   600
        with self.internal_session() as session:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   601
            for pk, version in session.execute(
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   602
                'Any K,V WHERE P is CWProperty, P value V, P pkey K, '
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   603
                'P pkey ~="system.version.%"', build_descr=False):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   604
                cube = pk.split('.')[-1]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   605
                # XXX cubicweb migration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   606
                if cube in CW_MIGRATION_MAP:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   607
                    cube = CW_MIGRATION_MAP[cube]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   608
                version = Version(version)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   609
                vcconf[cube] = version
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   610
                if checkversions:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   611
                    if cube != 'cubicweb':
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   612
                        fsversion = self.config.cube_version(cube)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   613
                    else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   614
                        fsversion = self.config.cubicweb_version()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   615
                    if version < fsversion:
2476
1294a6bdf3bf application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2473
diff changeset
   616
                        msg = ('instance has %s version %s but %s '
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   617
                               'is installed. Run "cubicweb-ctl upgrade".')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   618
                        raise ExecutionError(msg % (cube, version, fsversion))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   619
        return vcconf
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   620
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   621
    @cached
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   622
    def source_defs(self):
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   623
        """Return the a dictionary containing source uris as value and a
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   624
        dictionary describing each source as value.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   625
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   626
        This is a public method, not requiring a session id.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   627
        """
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   628
        sources = {}
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   629
        # remove sensitive information
6427
c8a5ac2d1eaa [schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6426
diff changeset
   630
        for uri, source in self.sources_by_uri.iteritems():
6722
3341521d857b [repo source] rename attribute so it's much easier to grasp its role
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6674
diff changeset
   631
            sources[uri] = source.public_config
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   632
        return sources
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   633
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   634
    def properties(self):
6256
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   635
        """Return a result set containing system wide properties.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   636
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   637
        This is a public method, not requiring a session id.
da77d3f95079 [repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6211
diff changeset
   638
        """
8433
ff9d6d269877 [server/session,repo] turn InternalSession, hence repo.internal_session, into a context manager (closes #2393651)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8393
diff changeset
   639
        with self.internal_session() as session:
5747
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   640
            # don't use session.execute, we don't want rset.req set
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   641
            return self.querier.execute(session, 'Any K,V WHERE P is CWProperty,'
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   642
                                        'P pkey K, P value V, NOT P for_user U',
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   643
                                        build_descr=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   644
5032
e3fa27fc0d9a cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5030
diff changeset
   645
    # XXX protect this method: anonymous should be allowed and registration
e3fa27fc0d9a cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5030
diff changeset
   646
    # plugged
1372
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   647
    def register_user(self, login, password, email=None, **kwargs):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   648
        """check a user with the given login exists, if not create it with the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   649
        given password. This method is designed to be used for anonymous
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   650
        registration on public web site.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   651
        """
8433
ff9d6d269877 [server/session,repo] turn InternalSession, hence repo.internal_session, into a context manager (closes #2393651)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8393
diff changeset
   652
        with self.internal_session() as session:
ff9d6d269877 [server/session,repo] turn InternalSession, hence repo.internal_session, into a context manager (closes #2393651)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8393
diff changeset
   653
            # for consistency, keep same error as unique check hook (although not required)
ff9d6d269877 [server/session,repo] turn InternalSession, hence repo.internal_session, into a context manager (closes #2393651)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8393
diff changeset
   654
            errmsg = session._('the value "%s" is already used, use another one')
5073
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   655
            if (session.execute('CWUser X WHERE X login %(login)s', {'login': login},
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   656
                                build_descr=False)
1398
5fe84a5f7035 rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents: 1372
diff changeset
   657
                or session.execute('CWUser X WHERE X use_email C, C address %(login)s',
5073
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   658
                                   {'login': login}, build_descr=False)):
5030
5238d9a8dfee [form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4984
diff changeset
   659
                qname = role_name('login', 'subject')
5238d9a8dfee [form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4984
diff changeset
   660
                raise ValidationError(None, {qname: errmsg % login})
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   661
            # we have to create the user
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
   662
            user = self.vreg['etypes'].etype_class('CWUser')(session)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   663
            if isinstance(password, unicode):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   664
                # password should *always* be utf8 encoded
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   665
                password = password.encode('UTF8')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   666
            kwargs['login'] = login
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   667
            kwargs['upassword'] = password
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
   668
            self.glob_add_entity(session, EditedEntity(user, **kwargs))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   669
            session.execute('SET X in_group G WHERE X eid %(x)s, G name "users"',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   670
                            {'x': user.eid})
1372
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   671
            if email or '@' in login:
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   672
                d = {'login': login, 'email': email or login}
5073
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   673
                if session.execute('EmailAddress X WHERE X address %(email)s', d,
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   674
                                   build_descr=False):
5030
5238d9a8dfee [form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4984
diff changeset
   675
                    qname = role_name('address', 'subject')
5238d9a8dfee [form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4984
diff changeset
   676
                    raise ValidationError(None, {qname: errmsg % d['email']})
1372
d4264cd876e1 register_user can now also set an email
Florent <florent@secondweb.fr>
parents: 1320
diff changeset
   677
                session.execute('INSERT EmailAddress X: X address %(email)s, '
5032
e3fa27fc0d9a cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5030
diff changeset
   678
                                'U primary_email X, U use_email X '
5073
a9697325cffa [repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5072
diff changeset
   679
                                'WHERE U login %(login)s', d, build_descr=False)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   680
            session.commit()
594
76218d42d21f return success or not on creation of user
Arthur Lutz <arthur.lutz@logilab.fr>
parents: 479
diff changeset
   681
        return True
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   682
6390
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   683
    def find_users(self, fetch_attrs, **query_attrs):
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   684
        """yield user attributes for cwusers matching the given query_attrs
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   685
        (the result set cannot survive this method call)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   686
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   687
        This can be used by low-privileges account (anonymous comes to
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   688
        mind).
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   689
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   690
        `fetch_attrs`: tuple of attributes to be fetched
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   691
        `query_attrs`: dict of attr/values to restrict the query
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   692
        """
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   693
        assert query_attrs
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   694
        if not hasattr(self, '_cwuser_attrs'):
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   695
            cwuser = self.schema['CWUser']
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   696
            self._cwuser_attrs = set(str(rschema)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   697
                                     for rschema, _eschema in cwuser.attribute_definitions()
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   698
                                     if not rschema.meta)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   699
        cwuserattrs = self._cwuser_attrs
8696
0bb18407c053 [toward py3k] rewrite dict.keys() and dict.values() (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8695
diff changeset
   700
        for k in chain(fetch_attrs, query_attrs):
6390
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   701
            if k not in cwuserattrs:
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   702
                raise Exception('bad input for find_user')
8433
ff9d6d269877 [server/session,repo] turn InternalSession, hence repo.internal_session, into a context manager (closes #2393651)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8393
diff changeset
   703
        with self.internal_session() as session:
6390
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   704
            varmaker = rqlvar_maker()
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   705
            vars = [(attr, varmaker.next()) for attr in fetch_attrs]
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   706
            rql = 'Any %s WHERE X is CWUser, ' % ','.join(var[1] for var in vars)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   707
            rql += ','.join('X %s %s' % (var[0], var[1]) for var in vars) + ','
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   708
            rset = session.execute(rql + ','.join('X %s %%(%s)s' % (attr, attr)
8696
0bb18407c053 [toward py3k] rewrite dict.keys() and dict.values() (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8695
diff changeset
   709
                                                  for attr in query_attrs),
6390
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   710
                                   query_attrs)
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   711
            return rset.rows
3766853656d7 [repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 6368
diff changeset
   712
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3629
diff changeset
   713
    def connect(self, login, **kwargs):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   714
        """open a connection for a given user
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   715
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   716
        raise `AuthenticationError` if the authentication failed
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   717
        raise `ConnectionError` if we can't open a connection
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   718
        """
8538
00597256de18 [request/session] refactor language handling: don't attempt to sync web/repo languages
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8537
diff changeset
   719
        cnxprops = kwargs.pop('cnxprops', None)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   720
        # use an internal connection
8433
ff9d6d269877 [server/session,repo] turn InternalSession, hence repo.internal_session, into a context manager (closes #2393651)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8393
diff changeset
   721
        with self.internal_session() as session:
ff9d6d269877 [server/session,repo] turn InternalSession, hence repo.internal_session, into a context manager (closes #2393651)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8393
diff changeset
   722
            # try to get a user object
3647
2941f4a0aab9 refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3629
diff changeset
   723
            user = self.authenticate_user(session, login, **kwargs)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   724
        session = Session(user, self, cnxprops)
9007
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
   725
        if threading.currentThread() in self._pyro_sessions:
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
   726
            # assume no pyro client does one get_repository followed by
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
   727
            # multiple repo.connect
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
   728
            assert self._pyro_sessions[threading.currentThread()] == None
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
   729
            self.debug('record session %s', session)
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
   730
            self._pyro_sessions[threading.currentThread()] = session
3379
9192ba07890d use .cw_rset instead of rset on appobject classes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3293
diff changeset
   731
        user._cw = user.cw_rset.req = session
5557
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
   732
        user.cw_clear_relation_cache()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   733
        self._sessions[session.id] = session
5638
cea831403e72 log the user name on session opening for easier debugging
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5603
diff changeset
   734
        self.info('opened session %s for user %s', session.id, login)
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
   735
        self.hm.call_hooks('session_open', session)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   736
        # commit session at this point in case write operation has been done
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   737
        # during `session_open` hooks
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   738
        session.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   739
        return session.id
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   740
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   741
    def execute(self, sessionid, rqlstring, args=None, build_descr=True,
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   742
                txid=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   743
        """execute a RQL query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   744
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   745
        * rqlstring should be an unicode string or a plain ascii string
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   746
        * args the optional parameters used in the query
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   747
        * build_descr is a flag indicating if the description should be
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   748
          built on select queries
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   749
        """
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   750
        session = self._get_session(sessionid, setcnxset=True, txid=txid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   751
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   752
            try:
5747
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   753
                rset = self.querier.execute(session, rqlstring, args,
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   754
                                            build_descr)
5747
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   755
                # NOTE: the web front will (re)build it when needed
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   756
                #       e.g in facets
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   757
                #       Zeroed to avoid useless overhead with pyro
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   758
                rset._rqlst = None
d6ac0cd30fde [rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 5711
diff changeset
   759
                return rset
9184
b982e88e4836 [repo] normalize ValidationError on edited entity (closes #2509729)
David Douard <david.douard@logilab.fr>
parents: 9171
diff changeset
   760
            except (ValidationError, Unauthorized, RQLSyntaxError):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   761
                raise
7616
8837e4733888 [repository] catch Exception instead of everything
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7573
diff changeset
   762
            except Exception:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   763
                # FIXME: check error to catch internal errors
5167
529861a73ec8 log RQL query in case of unexpected failure (priceless when porting to new database)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 5115
diff changeset
   764
                self.exception('unexpected error while executing %s with %s', rqlstring, args)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   765
                raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   766
        finally:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   767
            session.free_cnxset()
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   768
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   769
    def describe(self, sessionid, eid, txid=None):
7543
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
   770
        """return a tuple `(type, physical source uri, extid, actual source
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
   771
        uri)` for the entity of the given `eid`
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
   772
        """
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   773
        session = self._get_session(sessionid, setcnxset=True, txid=txid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   774
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   775
            return self.type_and_source_from_eid(eid, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   776
        finally:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   777
            session.free_cnxset()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   778
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   779
    def check_session(self, sessionid):
6012
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   780
        """raise `BadConnectionId` if the connection is no more valid, else
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   781
        return its latest activity timestamp.
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   782
        """
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   783
        return self._get_session(sessionid, setcnxset=False).timestamp
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   784
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   785
    def get_shared_data(self, sessionid, key, default=None, pop=False, txdata=False):
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   786
        """return value associated to key in the session's data dictionary or
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   787
        session's transaction's data if `txdata` is true.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   788
8238
087bb529035c [spelling] fix dictionnary -> dictionary typo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8224
diff changeset
   789
        If pop is True, value will be removed from the dictionary.
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   790
8238
087bb529035c [spelling] fix dictionnary -> dictionary typo
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8224
diff changeset
   791
        If key isn't defined in the dictionary, value specified by the
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   792
        `default` argument will be returned.
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   793
        """
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   794
        session = self._get_session(sessionid, setcnxset=False)
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   795
        return session.get_shared_data(key, default, pop, txdata)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   796
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   797
    def set_shared_data(self, sessionid, key, value, txdata=False):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   798
        """set value associated to `key` in shared data
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   799
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   800
        if `txdata` is true, the value will be added to the repository session's
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   801
        transaction's data which are cleared on commit/rollback of the current
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   802
        transaction.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   803
        """
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   804
        session = self._get_session(sessionid, setcnxset=False)
6013
8ca424bc393b [dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6012
diff changeset
   805
        session.set_shared_data(key, value, txdata)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   806
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   807
    def commit(self, sessionid, txid=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   808
        """commit transaction for the session with the given id"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   809
        self.debug('begin commit for session %s', sessionid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   810
        try:
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   811
            session = self._get_session(sessionid)
9020
cb87e831c183 rename server.session.transaction into server.session.connection
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9008
diff changeset
   812
            session.set_cnx(txid)
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   813
            return session.commit()
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   814
        except (ValidationError, Unauthorized):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   815
            raise
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   816
        except Exception:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   817
            self.exception('unexpected error')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   818
            raise
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   819
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   820
    def rollback(self, sessionid, txid=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   821
        """commit transaction for the session with the given id"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   822
        self.debug('begin rollback for session %s', sessionid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   823
        try:
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   824
            session = self._get_session(sessionid)
9020
cb87e831c183 rename server.session.transaction into server.session.connection
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9008
diff changeset
   825
            session.set_cnx(txid)
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   826
            session.rollback()
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   827
        except Exception:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   828
            self.exception('unexpected error')
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   829
            raise
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   830
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   831
    def close(self, sessionid, txid=None, checkshuttingdown=True):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   832
        """close the session with the given id"""
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   833
        session = self._get_session(sessionid, setcnxset=True, txid=txid,
1939
67e7379edd96 #343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1883
diff changeset
   834
                                    checkshuttingdown=checkshuttingdown)
9267
24d9b86dfa54 spelling: rollbacked -> rolled back
Rémi Cardona <remi.cardona@logilab.fr>
parents: 9226
diff changeset
   835
        # operation uncommited before close are rolled back before hook is called
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   836
        session.rollback(free_cnxset=False)
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
   837
        self.hm.call_hooks('session_close', session)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   838
        # commit session at this point in case write operation has been done
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   839
        # during `session_close` hooks
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   840
        session.commit()
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   841
        session.close()
9008
e0aa7cf8abf8 repository: make tests pass again
Julien Cristau <julien.cristau@logilab.fr>
parents: 9007
diff changeset
   842
        if threading.currentThread() in self._pyro_sessions:
e0aa7cf8abf8 repository: make tests pass again
Julien Cristau <julien.cristau@logilab.fr>
parents: 9007
diff changeset
   843
            self._pyro_sessions[threading.currentThread()] = None
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   844
        del self._sessions[sessionid]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   845
        self.info('closed session %s for user %s', sessionid, session.user.login)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   846
9033
614bf73cc126 [service] drop the asynchronous execution possibility
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9020
diff changeset
   847
    def call_service(self, sessionid, regid, **kwargs):
8268
c9babe49c1c1 [repository] implement a generic way to call repo-side services; closes #2203418
Florent Cayré <florent.cayre@logilab.fr>
parents: 8239
diff changeset
   848
        """
c9babe49c1c1 [repository] implement a generic way to call repo-side services; closes #2203418
Florent Cayré <florent.cayre@logilab.fr>
parents: 8239
diff changeset
   849
        See :class:`cubicweb.dbapi.Connection.call_service`
c9babe49c1c1 [repository] implement a generic way to call repo-side services; closes #2203418
Florent Cayré <florent.cayre@logilab.fr>
parents: 8239
diff changeset
   850
        and :class:`cubicweb.server.Service`
c9babe49c1c1 [repository] implement a generic way to call repo-side services; closes #2203418
Florent Cayré <florent.cayre@logilab.fr>
parents: 8239
diff changeset
   851
        """
9093
e2f88df79efd [connection] move call_service on Connection
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9035
diff changeset
   852
        # XXX lack a txid
8837
f7f4238ffe3c [service] extract session retrieval from "task" closure
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8699
diff changeset
   853
        session = self._get_session(sessionid)
9093
e2f88df79efd [connection] move call_service on Connection
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9035
diff changeset
   854
        return session._cnx.call_service(regid, **kwargs)
8268
c9babe49c1c1 [repository] implement a generic way to call repo-side services; closes #2203418
Florent Cayré <florent.cayre@logilab.fr>
parents: 8239
diff changeset
   855
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   856
    def user_info(self, sessionid, props=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   857
        """this method should be used by client to:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   858
        * check session id validity
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   859
        * update user information on each user's request (i.e. groups and
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   860
          custom properties)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   861
        """
8538
00597256de18 [request/session] refactor language handling: don't attempt to sync web/repo languages
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8537
diff changeset
   862
        user = self._get_session(sessionid, setcnxset=False).user
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   863
        return user.eid, user.login, user.groups, user.properties
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   864
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   865
    def undoable_transactions(self, sessionid, ueid=None, txid=None,
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   866
                              **actionfilters):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   867
        """See :class:`cubicweb.dbapi.Connection.undoable_transactions`"""
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   868
        session = self._get_session(sessionid, setcnxset=True, txid=txid)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   869
        try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   870
            return self.system_source.undoable_transactions(session, ueid,
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   871
                                                            **actionfilters)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   872
        finally:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   873
            session.free_cnxset()
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   874
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   875
    def transaction_info(self, sessionid, txuuid, txid=None):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   876
        """See :class:`cubicweb.dbapi.Connection.transaction_info`"""
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   877
        session = self._get_session(sessionid, setcnxset=True, txid=txid)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   878
        try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   879
            return self.system_source.tx_info(session, txuuid)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   880
        finally:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   881
            session.free_cnxset()
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   882
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   883
    def transaction_actions(self, sessionid, txuuid, public=True, txid=None):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   884
        """See :class:`cubicweb.dbapi.Connection.transaction_actions`"""
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   885
        session = self._get_session(sessionid, setcnxset=True, txid=txid)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   886
        try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   887
            return self.system_source.tx_actions(session, txuuid, public)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   888
        finally:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   889
            session.free_cnxset()
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   890
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   891
    def undo_transaction(self, sessionid, txuuid, txid=None):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   892
        """See :class:`cubicweb.dbapi.Connection.undo_transaction`"""
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   893
        session = self._get_session(sessionid, setcnxset=True, txid=txid)
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   894
        try:
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   895
            return self.system_source.undo_transaction(session, txuuid)
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   896
        finally:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   897
            session.free_cnxset()
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
   898
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   899
    # public (inter-repository) interface #####################################
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   900
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   901
    def entities_modified_since(self, etypes, mtime):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   902
        """function designed to be called from an external repository which
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   903
        is using this one as a rql source for synchronization, and return a
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   904
        3-uple containing :
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   905
        * the local date
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   906
        * list of (etype, eid) of entities of the given types which have been
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   907
          modified since the given timestamp (actually entities whose full text
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   908
          index content has changed)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   909
        * list of (etype, eid) of entities of the given types which have been
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   910
          deleted since the given timestamp
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   911
        """
8433
ff9d6d269877 [server/session,repo] turn InternalSession, hence repo.internal_session, into a context manager (closes #2393651)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8393
diff changeset
   912
        with self.internal_session() as session:
ff9d6d269877 [server/session,repo] turn InternalSession, hence repo.internal_session, into a context manager (closes #2393651)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8393
diff changeset
   913
            updatetime = datetime.utcnow()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   914
            modentities, delentities = self.system_source.modified_entities(
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   915
                session, etypes, mtime)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   916
            return updatetime, modentities, delentities
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   917
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   918
    # session handling ########################################################
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   919
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   920
    def close_sessions(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   921
        """close every opened sessions"""
8699
864ffd247d70 [towards py3k] fix bug introduced by 0bb18407c053
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8696
diff changeset
   922
        for sessionid in list(self._sessions):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   923
            try:
1939
67e7379edd96 #343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1883
diff changeset
   924
                self.close(sessionid, checkshuttingdown=False)
7815
2a164a9cf81c [exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7782
diff changeset
   925
            except Exception: # XXX BaseException?
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   926
                self.exception('error while closing session %s' % sessionid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   927
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   928
    def clean_sessions(self):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   929
        """close sessions not used since an amount of time specified in the
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   930
        configuration
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   931
        """
6012
d56fd78006cd [session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
   932
        mintime = time() - self.cleanup_session_time
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   933
        self.debug('cleaning session unused since %s',
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   934
                   strftime('%T', localtime(mintime)))
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   935
        nbclosed = 0
8890
57ffd0e0643d [clean_session] revert itervalues usage from 0bb18407c053
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8881
diff changeset
   936
        for session in self._sessions.values():
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   937
            if session.timestamp < mintime:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   938
                self.close(session.id)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   939
                nbclosed += 1
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   940
        return nbclosed
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   941
9404
3e3e9b37e177 fix version number: we still target cw 3.19
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9402
diff changeset
   942
    @deprecated("[3.19] use internal_cnx now\n"
9113
af6efc15fc90 [repository] add an ``internal_cnx`` method to replace ``internal_session``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9105
diff changeset
   943
                "(Beware that integrity hook are now enabled by default)")
7706
359bc86d2827 [session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7698
diff changeset
   944
    def internal_session(self, cnxprops=None, safe=False):
359bc86d2827 [session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7698
diff changeset
   945
        """return a dbapi like connection/cursor using internal user which have
359bc86d2827 [session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7698
diff changeset
   946
        every rights on the repository. The `safe` argument is a boolean flag
359bc86d2827 [session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7698
diff changeset
   947
        telling if integrity hooks should be activated or not.
359bc86d2827 [session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7698
diff changeset
   948
9127
aff75b69db92 [repository] drop safe attribute on ``internal_cnx``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9113
diff changeset
   949
        /!\ the safe argument is False by default.
aff75b69db92 [repository] drop safe attribute on ``internal_cnx``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9113
diff changeset
   950
7706
359bc86d2827 [session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7698
diff changeset
   951
        *YOU HAVE TO* commit/rollback or close (rollback implicitly) the
359bc86d2827 [session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7698
diff changeset
   952
        session once the job's done, else you'll leak connections set up to the
359bc86d2827 [session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7698
diff changeset
   953
        time where no one is available, causing irremediable freeze...
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   954
        """
7706
359bc86d2827 [session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7698
diff changeset
   955
        session = InternalSession(self, cnxprops, safe)
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   956
        session.set_cnxset()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   957
        return session
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   958
9113
af6efc15fc90 [repository] add an ``internal_cnx`` method to replace ``internal_session``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9105
diff changeset
   959
    @contextmanager
9127
aff75b69db92 [repository] drop safe attribute on ``internal_cnx``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9113
diff changeset
   960
    def internal_cnx(self):
9113
af6efc15fc90 [repository] add an ``internal_cnx`` method to replace ``internal_session``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9105
diff changeset
   961
        """return a Connection using internal user which have
9127
aff75b69db92 [repository] drop safe attribute on ``internal_cnx``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9113
diff changeset
   962
        every rights on the repository. The `safe` argument is dropped. all
aff75b69db92 [repository] drop safe attribute on ``internal_cnx``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9113
diff changeset
   963
        hook are enabled by default.
9113
af6efc15fc90 [repository] add an ``internal_cnx`` method to replace ``internal_session``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9105
diff changeset
   964
9127
aff75b69db92 [repository] drop safe attribute on ``internal_cnx``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9113
diff changeset
   965
        /!\ IN OPPOSITE OF THE OLDER INTERNAL_SESSION,
aff75b69db92 [repository] drop safe attribute on ``internal_cnx``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9113
diff changeset
   966
        /!\ INTERNAL CONNECTION HAVE ALL HOOKS ENABLED.
9113
af6efc15fc90 [repository] add an ``internal_cnx`` method to replace ``internal_session``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9105
diff changeset
   967
af6efc15fc90 [repository] add an ``internal_cnx`` method to replace ``internal_session``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9105
diff changeset
   968
        This is to be used a context manager.
af6efc15fc90 [repository] add an ``internal_cnx`` method to replace ``internal_session``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9105
diff changeset
   969
        """
9127
aff75b69db92 [repository] drop safe attribute on ``internal_cnx``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9113
diff changeset
   970
        with InternalSession(self) as session:
9113
af6efc15fc90 [repository] add an ``internal_cnx`` method to replace ``internal_session``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9105
diff changeset
   971
            with session.new_cnx() as cnx:
af6efc15fc90 [repository] add an ``internal_cnx`` method to replace ``internal_session``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9105
diff changeset
   972
                yield cnx
af6efc15fc90 [repository] add an ``internal_cnx`` method to replace ``internal_session``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9105
diff changeset
   973
af6efc15fc90 [repository] add an ``internal_cnx`` method to replace ``internal_session``
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9105
diff changeset
   974
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   975
    def _get_session(self, sessionid, setcnxset=False, txid=None,
5813
0b250d72fcfa [transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5808
diff changeset
   976
                     checkshuttingdown=True):
9272
68744f5154c4 [server] fix a number of typos, mostly in docstrings
Julien Cristau <julien.cristau@logilab.fr>
parents: 9267
diff changeset
   977
        """return the session associated with the given session identifier"""
5906
d40ced753291 [repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5813
diff changeset
   978
        if checkshuttingdown and self.shutting_down:
7573
c8f8762c986d [repo, looping task] raise a custom exception when repository is shuting down, avoid looping task to be restarted in such case. Closes #1021276
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7550
diff changeset
   979
            raise ShuttingDown('Repository is shutting down')
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   980
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   981
            session = self._sessions[sessionid]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   982
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   983
            raise BadConnectionId('No such session %s' % sessionid)
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   984
        if setcnxset:
9020
cb87e831c183 rename server.session.transaction into server.session.connection
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 9008
diff changeset
   985
            session.set_cnx(txid) # must be done before set_cnxset
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
   986
            session.set_cnxset()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   987
        return session
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   988
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   989
    # data sources handling ###################################################
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   990
    # * correspondance between eid and (type, source)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   991
    # * correspondance between eid and local id (i.e. specific to a given source)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
   992
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   993
    def type_and_source_from_eid(self, eid, session=None):
7543
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
   994
        """return a tuple `(type, physical source uri, extid, actual source
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
   995
        uri)` for the entity of the given `eid`
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
   996
        """
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   997
        try:
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8699
diff changeset
   998
            eid = int(eid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
   999
        except ValueError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1000
            raise UnknownEid(eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1001
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1002
            return self._type_source_cache[eid]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1003
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1004
            if session is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1005
                session = self.internal_session()
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1006
                free_cnxset = True
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1007
            else:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1008
                free_cnxset = False
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1009
            try:
7543
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
  1010
                etype, uri, extid, auri = self.system_source.eid_type_source(
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
  1011
                    session, eid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1012
            finally:
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1013
                if free_cnxset:
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1014
                    session.free_cnxset()
7543
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
  1015
            self._type_source_cache[eid] = (etype, uri, extid, auri)
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
  1016
            if uri != 'system':
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
  1017
                self._extid_cache[(extid, uri)] = eid
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
  1018
            return etype, uri, extid, auri
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1019
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1020
    def clear_caches(self, eids):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1021
        etcache = self._type_source_cache
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1022
        extidcache = self._extid_cache
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1023
        rqlcache = self.querier._rql_cache
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1024
        for eid in eids:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1025
            try:
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8699
diff changeset
  1026
                etype, uri, extid, auri = etcache.pop(int(eid)) # may be a string in some cases
8133
b0a70092946a [repo cache] fix cache clearing for an eid: keys are now always a tuple
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8074
diff changeset
  1027
                rqlcache.pop( ('%s X WHERE X eid %s' % (etype, eid),), None)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1028
                extidcache.pop((extid, uri), None)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1029
            except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1030
                etype = None
8133
b0a70092946a [repo cache] fix cache clearing for an eid: keys are now always a tuple
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8074
diff changeset
  1031
            rqlcache.pop( ('Any X WHERE X eid %s' % eid,), None)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1032
            for source in self.sources:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1033
                source.clear_eid_cache(eid, etype)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1034
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1035
    def type_from_eid(self, eid, session=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1036
        """return the type of the entity with id <eid>"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1037
        return self.type_and_source_from_eid(eid, session)[0]
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1038
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1039
    def source_from_eid(self, eid, session=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1040
        """return the source for the given entity's eid"""
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1041
        return self.sources_by_uri[self.type_and_source_from_eid(eid, session)[1]]
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1042
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1043
    def querier_cache_key(self, session, rql, args, eidkeys):
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1044
        cachekey = [rql]
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1045
        for key in sorted(eidkeys):
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1046
            try:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1047
                etype = self.type_from_eid(args[key], session)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1048
            except KeyError:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1049
                raise QueryError('bad cache key %s (no value)' % key)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1050
            except TypeError:
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1051
                raise QueryError('bad cache key %s (value: %r)' % (
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1052
                    key, args[key]))
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1053
            cachekey.append(etype)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1054
            # ensure eid is correctly typed in args
8748
f5027f8d2478 drop typed_eid() in favour of int() (closes #2742462)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8699
diff changeset
  1055
            args[key] = int(args[key])
5174
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1056
        return tuple(cachekey)
78438ad513ca #759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5115
diff changeset
  1057
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1058
    def eid2extid(self, source, eid, session=None):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1059
        """get local id from an eid"""
7543
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
  1060
        etype, uri, extid, _ = self.type_and_source_from_eid(eid, session)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1061
        if source.uri != uri:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1062
            # eid not from the given source
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1063
            raise UnknownEid(eid)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1064
        return extid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1065
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1066
    def extid2eid(self, source, extid, etype, session=None, insert=True,
7708
45be3a9debe6 [datafeed] for datafeed source, we don't want commit in extid2eid but explicitly handled by the source. Also, we should use 'safe' internal session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7706
diff changeset
  1067
                  complete=True, commit=True, sourceparams=None):
7399
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1068
        """Return eid from a local id. If the eid is a negative integer, that
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1069
        means the entity is known but has been copied back to the system source
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1070
        hence should be ignored.
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1071
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1072
        If no record is found, ie the entity is not known yet:
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1073
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1074
        1. an eid is attributed
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1075
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1076
        2. the source's :meth:`before_entity_insertion` method is called to
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1077
           build the entity instance
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1078
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1079
        3. unless source's :attr:`should_call_hooks` tell otherwise,
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1080
          'before_add_entity' hooks are called
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1081
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1082
        4. record is added into the system source
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1083
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1084
        5. the source's :meth:`after_entity_insertion` method is called to
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1085
           complete building of the entity instance
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1086
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1087
        6. unless source's :attr:`should_call_hooks` tell otherwise,
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1088
          'before_add_entity' hooks are called
972ed1843bd8 [multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7398
diff changeset
  1089
        """
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1090
        uri = 'system' if source.copy_based_source else source.uri
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1091
        cachekey = (extid, uri)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1092
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1093
            return self._extid_cache[cachekey]
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1094
        except KeyError:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1095
            pass
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1096
        free_cnxset = False
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1097
        if session is None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1098
            session = self.internal_session()
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1099
            free_cnxset = True
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1100
        eid = self.system_source.extid2eid(session, uri, extid)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1101
        if eid is not None:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1102
            self._extid_cache[cachekey] = eid
7543
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
  1103
            self._type_source_cache[eid] = (etype, uri, extid, source.uri)
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1104
            if free_cnxset:
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1105
                session.free_cnxset()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1106
            return eid
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1107
        if not insert:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1108
            return
1954
9b20f3504af8 cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 1939
diff changeset
  1109
        # no link between extid and eid, create one using an internal session
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1110
        # since the current session user may not have required permissions to
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1111
        # do necessary stuff and we don't want to commit user session.
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1112
        #
450
5e14ea0e81c8 a note for later
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 341
diff changeset
  1113
        # Moreover, even if session is already an internal session but is
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1114
        # processing a commit, we have to use another one
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1115
        if not session.is_internal_session:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1116
            session = self.internal_session()
7398
26695dd703d8 [repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1117
            free_cnxset = True
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1118
        try:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1119
            eid = self.system_source.create_eid(session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1120
            self._extid_cache[cachekey] = eid
7543
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
  1121
            self._type_source_cache[eid] = (etype, uri, extid, source.uri)
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1122
            entity = source.before_entity_insertion(
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1123
                session, extid, etype, eid, sourceparams)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1124
            if source.should_call_hooks:
7950
99ae8c883ad3 [ms repository] when an exception is raised during extid2eid and no rollback is done, some manual cleanups have to be done (closes #1993420)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7931
diff changeset
  1125
                # get back a copy of operation for later restore if necessary,
99ae8c883ad3 [ms repository] when an exception is raised during extid2eid and no rollback is done, some manual cleanups have to be done (closes #1993420)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7931
diff changeset
  1126
                # see below
99ae8c883ad3 [ms repository] when an exception is raised during extid2eid and no rollback is done, some manual cleanups have to be done (closes #1993420)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7931
diff changeset
  1127
                pending_operations = session.pending_operations[:]
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1128
                self.hm.call_hooks('before_add_entity', session, entity=entity)
7698
1c7411535c2d [datafeed / fti] rather control a 'complete' parameter than setting empty attribute values
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7622
diff changeset
  1129
            self.add_info(session, entity, source, extid, complete=complete)
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1130
            source.after_entity_insertion(session, extid, entity, sourceparams)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1131
            if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1132
                self.hm.call_hooks('after_add_entity', session, entity=entity)
7716
c893150b2394 [repo ms] commit/rollback session if it has been created in the method
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7708
diff changeset
  1133
            if commit or free_cnxset:
7708
45be3a9debe6 [datafeed] for datafeed source, we don't want commit in extid2eid but explicitly handled by the source. Also, we should use 'safe' internal session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7706
diff changeset
  1134
                session.commit(free_cnxset)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1135
            return eid
7708
45be3a9debe6 [datafeed] for datafeed source, we don't want commit in extid2eid but explicitly handled by the source. Also, we should use 'safe' internal session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7706
diff changeset
  1136
        except Exception:
7716
c893150b2394 [repo ms] commit/rollback session if it has been created in the method
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7708
diff changeset
  1137
            if commit or free_cnxset:
7708
45be3a9debe6 [datafeed] for datafeed source, we don't want commit in extid2eid but explicitly handled by the source. Also, we should use 'safe' internal session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7706
diff changeset
  1138
                session.rollback(free_cnxset)
7950
99ae8c883ad3 [ms repository] when an exception is raised during extid2eid and no rollback is done, some manual cleanups have to be done (closes #1993420)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7931
diff changeset
  1139
            else:
99ae8c883ad3 [ms repository] when an exception is raised during extid2eid and no rollback is done, some manual cleanups have to be done (closes #1993420)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7931
diff changeset
  1140
                # XXX do some cleanup manually so that the transaction has a
99ae8c883ad3 [ms repository] when an exception is raised during extid2eid and no rollback is done, some manual cleanups have to be done (closes #1993420)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7931
diff changeset
  1141
                # chance to be commited, with simply this entity discarded
99ae8c883ad3 [ms repository] when an exception is raised during extid2eid and no rollback is done, some manual cleanups have to be done (closes #1993420)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7931
diff changeset
  1142
                self._extid_cache.pop(cachekey, None)
99ae8c883ad3 [ms repository] when an exception is raised during extid2eid and no rollback is done, some manual cleanups have to be done (closes #1993420)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7931
diff changeset
  1143
                self._type_source_cache.pop(eid, None)
99ae8c883ad3 [ms repository] when an exception is raised during extid2eid and no rollback is done, some manual cleanups have to be done (closes #1993420)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7931
diff changeset
  1144
                if 'entity' in locals():
99ae8c883ad3 [ms repository] when an exception is raised during extid2eid and no rollback is done, some manual cleanups have to be done (closes #1993420)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7931
diff changeset
  1145
                    hook.CleanupDeletedEidsCacheOp.get_instance(session).add_data(entity.eid)
99ae8c883ad3 [ms repository] when an exception is raised during extid2eid and no rollback is done, some manual cleanups have to be done (closes #1993420)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7931
diff changeset
  1146
                    self.system_source.delete_info_multi(session, [entity], uri)
99ae8c883ad3 [ms repository] when an exception is raised during extid2eid and no rollback is done, some manual cleanups have to be done (closes #1993420)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7931
diff changeset
  1147
                    if source.should_call_hooks:
8773
21edcb0a5ed7 [session] rename `_threaddata` to `_tx`
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8771
diff changeset
  1148
                        session._tx.pending_operations = pending_operations
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1149
            raise
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1150
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1151
    def add_info(self, session, entity, source, extid=None, complete=True):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1152
        """add type and source info for an eid into the system table,
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1153
        and index the entity with the full text index
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1154
        """
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1155
        # begin by inserting eid/type/source/extid into the entities table
6426
541659c39f6a [hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6390
diff changeset
  1156
        hook.CleanupNewEidsCacheOp.get_instance(session).add_data(entity.eid)
4806
4f12f59b1a13 [fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4756
diff changeset
  1157
        self.system_source.add_info(session, entity, source, extid, complete)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1158
7884
35d2e2f4e10a [repo] cleanup [_]delete_info prototype: extid argument isn't used
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
  1159
    def delete_info(self, session, entity, sourceuri, scleanup=None):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1160
        """called by external source when some entity known by the system source
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1161
        has been deleted in the external source
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1162
        """
5067
adc2122eed03 [repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5043
diff changeset
  1163
        # mark eid as being deleted in session info and setup cache update
adc2122eed03 [repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5043
diff changeset
  1164
        # operation
6426
541659c39f6a [hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6390
diff changeset
  1165
        hook.CleanupDeletedEidsCacheOp.get_instance(session).add_data(entity.eid)
7884
35d2e2f4e10a [repo] cleanup [_]delete_info prototype: extid argument isn't used
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
  1166
        self._delete_info(session, entity, sourceuri, scleanup)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1167
7884
35d2e2f4e10a [repo] cleanup [_]delete_info prototype: extid argument isn't used
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
  1168
    def _delete_info(self, session, entity, sourceuri, scleanup=None):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1169
        """delete system information on deletion of an entity:
7884
35d2e2f4e10a [repo] cleanup [_]delete_info prototype: extid argument isn't used
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
  1170
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1171
        * delete all remaining relations from/to this entity
7884
35d2e2f4e10a [repo] cleanup [_]delete_info prototype: extid argument isn't used
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
  1172
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1173
        * call delete info on the system source which will transfer record from
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1174
          the entities table to the deleted_entities table
7884
35d2e2f4e10a [repo] cleanup [_]delete_info prototype: extid argument isn't used
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
  1175
35d2e2f4e10a [repo] cleanup [_]delete_info prototype: extid argument isn't used
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
  1176
        When scleanup is specified, it's expected to be the source's eid, in
35d2e2f4e10a [repo] cleanup [_]delete_info prototype: extid argument isn't used
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
  1177
        which case we'll specify the target's relation source so that this
35d2e2f4e10a [repo] cleanup [_]delete_info prototype: extid argument isn't used
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
  1178
        source is ignored. E.g. we want to delete relations stored locally, as
35d2e2f4e10a [repo] cleanup [_]delete_info prototype: extid argument isn't used
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
  1179
        the deletion information comes from the external source, it's its
35d2e2f4e10a [repo] cleanup [_]delete_info prototype: extid argument isn't used
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7823
diff changeset
  1180
        responsability to have cleaned-up its own relations.
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1181
        """
2641
9c33d98a074e R [schema hooks] big refactoring / reorganization for clearer code, a few fixes on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2631
diff changeset
  1182
        pendingrtypes = session.transaction_data.get('pendingrtypes', ())
7887
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1183
        if scleanup is not None:
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1184
            source = self.sources_by_eid[scleanup]
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1185
        # delete remaining relations: if user can delete the entity, he can
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1186
        # delete all its relations without security checking
8562
0d2fb4604265 [session] fix arguments default value and promote usage of security_enabled as session method. Closes #2481820
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8539
diff changeset
  1187
        with session.security_enabled(read=False, write=False):
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1188
            eid = entity.eid
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1189
            for rschema, _, role in entity.e_schema.relation_definitions():
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1190
                rtype = rschema.type
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1191
                if rtype in schema.VIRTUAL_RTYPES or rtype in pendingrtypes:
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1192
                    continue
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1193
                if role == 'subject':
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1194
                    # don't skip inlined relation so they are regularly
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1195
                    # deleted and so hooks are correctly called
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
  1196
                    rql = 'DELETE X %s Y WHERE X eid %%(x)s' % rtype
4835
13b0b96d7982 [repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4834
diff changeset
  1197
                else:
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
  1198
                    rql = 'DELETE Y %s X WHERE X eid %%(x)s' % rtype
6941
9ed02daa7dbb [repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
  1199
                if scleanup is not None:
7887
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1200
                    # if the relation can't be crossed, nothing to cleanup (we
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1201
                    # would get a BadRQLQuery from the multi-sources planner).
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1202
                    # This may still leave some junk if the mapping has changed
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1203
                    # at some point, but one can still run db-check to catch
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1204
                    # those
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1205
                    if not source in self.can_cross_relation(rtype):
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1206
                        continue
6626
5c20a005bddc [pyro source] when cleaning relations of an external entity being cleaned up, only delete local relationsd
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6491
diff changeset
  1207
                    # source cleaning: only delete relations stored locally
6941
9ed02daa7dbb [repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
  1208
                    # (here, scleanup
9ed02daa7dbb [repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
  1209
                    rql += ', NOT (Y cw_source S, S eid %(seid)s)'
6674
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1210
                try:
6941
9ed02daa7dbb [repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
  1211
                    session.execute(rql, {'x': eid, 'seid': scleanup},
6674
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1212
                                    build_descr=False)
7616
8837e4733888 [repository] catch Exception instead of everything
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7573
diff changeset
  1213
                except Exception:
7885
9454b7ef5ae4 [ms, test] propagate exception in test mode
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7884
diff changeset
  1214
                    if self.config.mode == 'test':
9454b7ef5ae4 [ms, test] propagate exception in test mode
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7884
diff changeset
  1215
                        raise
6674
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1216
                    self.exception('error while cascading delete for entity %s '
fd9c76196bf7 [repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6664
diff changeset
  1217
                                   'from %s. RQL: %s', entity, sourceuri, rql)
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1218
        self.system_source.delete_info_multi(session, [entity], sourceuri)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1219
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1220
    def _delete_info_multi(self, session, entities, sourceuri, scleanup=None):
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1221
        """same as _delete_info but accepts a list of entities with
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1222
        the same etype and belinging to the same source.
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1223
        """
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1224
        pendingrtypes = session.transaction_data.get('pendingrtypes', ())
7887
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1225
        if scleanup is not None:
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1226
            source = self.sources_by_eid[scleanup]
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1227
        # delete remaining relations: if user can delete the entity, he can
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1228
        # delete all its relations without security checking
8562
0d2fb4604265 [session] fix arguments default value and promote usage of security_enabled as session method. Closes #2481820
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8539
diff changeset
  1229
        with session.security_enabled(read=False, write=False):
7895
0a967180794b cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7887
diff changeset
  1230
            in_eids = ','.join([str(_e.eid) for _e in entities])
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1231
            for rschema, _, role in entities[0].e_schema.relation_definitions():
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1232
                rtype = rschema.type
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1233
                if rtype in schema.VIRTUAL_RTYPES or rtype in pendingrtypes:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1234
                    continue
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1235
                if role == 'subject':
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1236
                    # don't skip inlined relation so they are regularly
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1237
                    # deleted and so hooks are correctly called
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1238
                    rql = 'DELETE X %s Y WHERE X eid IN (%s)' % (rtype, in_eids)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1239
                else:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1240
                    rql = 'DELETE Y %s X WHERE X eid IN (%s)' % (rtype, in_eids)
6941
9ed02daa7dbb [repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
  1241
                if scleanup is not None:
7887
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1242
                    # if the relation can't be crossed, nothing to cleanup (we
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1243
                    # would get a BadRQLQuery from the multi-sources planner).
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1244
                    # This may still leave some junk if the mapping has changed
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1245
                    # at some point, but one can still run db-check to catch
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1246
                    # those
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1247
                    if not source in self.can_cross_relation(rtype):
42a0b7398d31 [repository] when deleting relations of a deleted external entity, we should skip non-crossable relation (closes #1973680)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7885
diff changeset
  1248
                        continue
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1249
                    # source cleaning: only delete relations stored locally
6941
9ed02daa7dbb [repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
  1250
                    rql += ', NOT (Y cw_source S, S eid %(seid)s)'
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1251
                try:
6941
9ed02daa7dbb [repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6931
diff changeset
  1252
                    session.execute(rql, {'seid': scleanup}, build_descr=False)
8051
41398fa1a90f [rql, integrity] let ValidationError and Unauthorized pass during composite deletion (closes #2070042)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7950
diff changeset
  1253
                except ValidationError:
41398fa1a90f [rql, integrity] let ValidationError and Unauthorized pass during composite deletion (closes #2070042)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7950
diff changeset
  1254
                    raise
41398fa1a90f [rql, integrity] let ValidationError and Unauthorized pass during composite deletion (closes #2070042)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7950
diff changeset
  1255
                except Unauthorized:
41398fa1a90f [rql, integrity] let ValidationError and Unauthorized pass during composite deletion (closes #2070042)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7950
diff changeset
  1256
                    self.exception('Unauthorized exception while cascading delete for entity %s '
41398fa1a90f [rql, integrity] let ValidationError and Unauthorized pass during composite deletion (closes #2070042)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7950
diff changeset
  1257
                                   'from %s. RQL: %s.\nThis should not happen since security is disabled here.',
41398fa1a90f [rql, integrity] let ValidationError and Unauthorized pass during composite deletion (closes #2070042)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7950
diff changeset
  1258
                                   entities, sourceuri, rql)
41398fa1a90f [rql, integrity] let ValidationError and Unauthorized pass during composite deletion (closes #2070042)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7950
diff changeset
  1259
                    raise
7616
8837e4733888 [repository] catch Exception instead of everything
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7573
diff changeset
  1260
                except Exception:
7885
9454b7ef5ae4 [ms, test] propagate exception in test mode
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7884
diff changeset
  1261
                    if self.config.mode == 'test':
9454b7ef5ae4 [ms, test] propagate exception in test mode
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7884
diff changeset
  1262
                        raise
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1263
                    self.exception('error while cascading delete for entity %s '
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1264
                                   'from %s. RQL: %s', entities, sourceuri, rql)
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1265
        self.system_source.delete_info_multi(session, entities, sourceuri)
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1266
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1267
    def locate_relation_source(self, session, subject, rtype, object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1268
        subjsource = self.source_from_eid(subject, session)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1269
        objsource = self.source_from_eid(object, session)
3042
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1270
        if not subjsource is objsource:
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1271
            source = self.system_source
4913
083b4d454192 server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents: 4899
diff changeset
  1272
            if not (subjsource.may_cross_relation(rtype)
3042
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1273
                    and objsource.may_cross_relation(rtype)):
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1274
                raise MultiSourcesError(
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1275
                    "relation %s can't be crossed among sources"
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1276
                    % rtype)
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1277
        elif not subjsource.support_relation(rtype):
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1278
            source = self.system_source
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1279
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1280
            source = subjsource
3042
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1281
        if not source.support_relation(rtype, True):
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1282
            raise MultiSourcesError(
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1283
                "source %s doesn't support write of %s relation"
d2455badf7fb [multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2967
diff changeset
  1284
                % (source.uri, rtype))
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1285
        return source
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1286
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1287
    def locate_etype_source(self, etype):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1288
        for source in self.sources:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1289
            if source.support_entity(etype, 1):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1290
                return source
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1291
        else:
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1292
            raise ETypeNotSupportedBySources(etype)
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1293
5068
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1294
    def init_entity_caches(self, session, entity, source):
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1295
        """add entity to session entities cache and repo's extid cache.
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1296
        Return entity's ext id if the source isn't the system source.
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1297
        """
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1298
        session.set_entity_cache(entity)
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1299
        suri = source.uri
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1300
        if suri == 'system':
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1301
            extid = None
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1302
        else:
6957
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1303
            if source.copy_based_source:
ffda12be2e9f [repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6953
diff changeset
  1304
                suri = 'system'
5068
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1305
            extid = source.get_extid(entity)
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1306
            self._extid_cache[(str(extid), suri)] = entity.eid
8900
010a59e12d89 use cw_etype instead of __regid__
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8889
diff changeset
  1307
        self._type_source_cache[entity.eid] = (entity.cw_etype, suri, extid,
7543
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
  1308
                                               source.uri)
5068
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1309
        return extid
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1310
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1311
    def glob_add_entity(self, session, edited):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1312
        """add an entity to the repository
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1313
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1314
        the entity eid should originaly be None and a unique eid is assigned to
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1315
        the entity instance
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1316
        """
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1317
        entity = edited.entity
5557
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1318
        entity._cw_is_saved = False # entity has an eid but is not yet saved
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1319
        # init edited_attributes before calling before_add_entity hooks
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1320
        entity.cw_edited = edited
8900
010a59e12d89 use cw_etype instead of __regid__
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8889
diff changeset
  1321
        source = self.locate_etype_source(entity.cw_etype)
5072
072ae171aeb0 [cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5068
diff changeset
  1322
        # allocate an eid to the entity before calling hooks
5557
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1323
        entity.eid = self.system_source.create_eid(session)
5068
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1324
        # set caches asap
10c3422d7419 [repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5067
diff changeset
  1325
        extid = self.init_entity_caches(session, entity, source)
2600
6cd6c5d11b45 [F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
  1326
        if server.DEBUG & server.DBG_REPO:
8900
010a59e12d89 use cw_etype instead of __regid__
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8889
diff changeset
  1327
            print 'ADD entity', self, entity.cw_etype, entity.eid, edited
8456
c912d82f2166 [repository] drop unused argument
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8433
diff changeset
  1328
        prefill_entity_caches(entity)
2929
51cdfe069e10 fix edited_attributes handling when adding entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2875
diff changeset
  1329
        if source.should_call_hooks:
2968
0e3460341023 somewhat painful backport of 3.5 branch, should mostly be ok
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2893 2967
diff changeset
  1330
            self.hm.call_hooks('before_add_entity', session, entity=entity)
8547
f23ac525ddd1 [datafeed] properly call hooks for inlined relations on entity creation. Closes #2481156
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8456
diff changeset
  1331
        relations = preprocess_inlined_relations(session, entity)
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1332
        edited.set_defaults()
4843
5f7363416765 fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4835
diff changeset
  1333
        if session.is_hook_category_activated('integrity'):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1334
            edited.check(creation=True)
6211
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1335
        try:
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1336
            source.add_entity(session, entity)
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8694
diff changeset
  1337
        except UniqueTogetherError as exc:
6465
6401a9d0b5aa [architecture] introduce adapter to ease transformation of errors before display to the end user. Use it for UniqueTogetherError first
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6464
diff changeset
  1338
            userhdlr = session.vreg['adapters'].select(
6401a9d0b5aa [architecture] introduce adapter to ease transformation of errors before display to the end user. Use it for UniqueTogetherError first
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6464
diff changeset
  1339
                'IUserFriendlyError', session, entity=entity, exc=exc)
6401a9d0b5aa [architecture] introduce adapter to ease transformation of errors before display to the end user. Use it for UniqueTogetherError first
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6464
diff changeset
  1340
            userhdlr.raise_user_exception()
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1341
        self.add_info(session, entity, source, extid, complete=False)
6368
f907cc7f2875 [repo] properly mark object as saved once added to its source (code much probably gone during a merge...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6333
diff changeset
  1342
        edited.saved = entity._cw_is_saved = True
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1343
        # trigger after_add_entity after after_add_relation
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1344
        if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1345
            self.hm.call_hooks('after_add_entity', session, entity=entity)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1346
            # call hooks for inlined relations
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1347
            for attr, value in relations:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1348
                self.hm.call_hooks('before_add_relation', session,
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1349
                                    eidfrom=entity.eid, rtype=attr, eidto=value)
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1350
                self.hm.call_hooks('after_add_relation', session,
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1351
                                    eidfrom=entity.eid, rtype=attr, eidto=value)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1352
        return entity.eid
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1353
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1354
    def glob_update_entity(self, session, edited):
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1355
        """replace an entity in the repository
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1356
        the type and the eid of an entity must not be changed
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1357
        """
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1358
        entity = edited.entity
2600
6cd6c5d11b45 [F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
  1359
        if server.DEBUG & server.DBG_REPO:
8900
010a59e12d89 use cw_etype instead of __regid__
Pierre-Yves David <pierre-yves.david@logilab.fr>
parents: 8889
diff changeset
  1360
            print 'UPDATE entity', entity.cw_etype, entity.eid, \
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1361
                  entity.cw_attr_cache, edited
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1362
        hm = self.hm
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1363
        eschema = entity.e_schema
2647
b0a2e779845c enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2641
diff changeset
  1364
        session.set_entity_cache(entity)
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1365
        orig_edited = getattr(entity, 'cw_edited', None)
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1366
        entity.cw_edited = edited
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1367
        try:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1368
            only_inline_rels, need_fti_update = True, False
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1369
            relations = []
5292
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1370
            source = self.source_from_eid(entity.eid, session)
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1371
            for attr in list(edited):
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1372
                if attr == 'eid':
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1373
                    continue
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1374
                rschema = eschema.subjrels[attr]
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1375
                if rschema.final:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1376
                    if getattr(eschema.rdef(attr), 'fulltextindexed', False):
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1377
                        need_fti_update = True
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1378
                    only_inline_rels = False
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1379
                else:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1380
                    # inlined relation
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1381
                    previous_value = entity.related(attr) or None
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1382
                    if previous_value is not None:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1383
                        previous_value = previous_value[0][0] # got a result set
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1384
                        if previous_value == entity.cw_attr_cache[attr]:
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1385
                            previous_value = None
5292
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1386
                        elif source.should_call_hooks:
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1387
                            hm.call_hooks('before_delete_relation', session,
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1388
                                          eidfrom=entity.eid, rtype=attr,
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1389
                                          eidto=previous_value)
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1390
                    relations.append((attr, edited[attr], previous_value))
5292
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1391
            if source.should_call_hooks:
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1392
                # call hooks for inlined relations
6211
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1393
                for attr, value, _t in relations:
5292
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1394
                    hm.call_hooks('before_add_relation', session,
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1395
                                  eidfrom=entity.eid, rtype=attr, eidto=value)
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1396
                if not only_inline_rels:
8b496574b65c [repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5187
diff changeset
  1397
                    hm.call_hooks('before_update_entity', session, entity=entity)
5808
2de32c0c293b [repo] call entity.check() once before_update hooks has been called, to let them a chance to transform data
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5749
diff changeset
  1398
            if session.is_hook_category_activated('integrity'):
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1399
                edited.check()
6211
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1400
            try:
e9d125fd1465 nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6189
diff changeset
  1401
                source.update_entity(session, entity)
6225
a176e68b7d0d backport stable
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6142 6211
diff changeset
  1402
                edited.saved = True
8695
358d8bed9626 [toward-py3k] rewrite to "except AnException as exc:" (part of #2711624)
Nicolas Chauvat <nicolas.chauvat@logilab.fr>
parents: 8694
diff changeset
  1403
            except UniqueTogetherError as exc:
9226
653f1d4a1101 [repository] properly use IUserFriendlyError when UniqueTogetherError is raised during entity update. Closes #3096638
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9184
diff changeset
  1404
                userhdlr = session.vreg['adapters'].select(
653f1d4a1101 [repository] properly use IUserFriendlyError when UniqueTogetherError is raised during entity update. Closes #3096638
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9184
diff changeset
  1405
                    'IUserFriendlyError', session, entity=entity, exc=exc)
653f1d4a1101 [repository] properly use IUserFriendlyError when UniqueTogetherError is raised during entity update. Closes #3096638
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 9184
diff changeset
  1406
                userhdlr.raise_user_exception()
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1407
            self.system_source.update_info(session, entity, need_fti_update)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1408
            if source.should_call_hooks:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1409
                if not only_inline_rels:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1410
                    hm.call_hooks('after_update_entity', session, entity=entity)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1411
                for attr, value, prevvalue in relations:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1412
                    # if the relation is already cached, update existant cache
5557
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1413
                    relcache = entity.cw_relation_cached(attr, 'subject')
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1414
                    if prevvalue is not None:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1415
                        hm.call_hooks('after_delete_relation', session,
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1416
                                      eidfrom=entity.eid, rtype=attr, eidto=prevvalue)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1417
                        if relcache is not None:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1418
                            session.update_rel_cache_del(entity.eid, attr, prevvalue)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1419
                    del_existing_rel_if_needed(session, entity.eid, attr, value)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1420
                    if relcache is not None:
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1421
                        session.update_rel_cache_add(entity.eid, attr, value)
2647
b0a2e779845c enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2641
diff changeset
  1422
                    else:
5557
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1423
                        entity.cw_set_relation_cache(attr, 'subject',
1a534c596bff [entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5556
diff changeset
  1424
                                                     session.eid_rset(value))
5115
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1425
                    hm.call_hooks('after_add_relation', session,
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1426
                                  eidfrom=entity.eid, rtype=attr, eidto=value)
2e43ef618d14 [repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5093
diff changeset
  1427
        finally:
6142
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1428
            if orig_edited is not None:
8bc6eac1fac1 [session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6139
diff changeset
  1429
                entity.cw_edited = orig_edited
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1430
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1431
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1432
    def glob_delete_entities(self, session, eids):
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1433
        """delete a list of  entities and all related entities from the repository"""
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1434
        # mark eids as being deleted in session info and setup cache update
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1435
        # operation (register pending eids before actual deletion to avoid
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1436
        # multiple call to glob_delete_entities)
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1437
        op = hook.CleanupDeletedEidsCacheOp.get_instance(session)
7507
4c043afb104a fix failures introduced by recent refactoring
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7503
diff changeset
  1438
        if not isinstance(eids, (set, frozenset)):
4c043afb104a fix failures introduced by recent refactoring
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7503
diff changeset
  1439
            warn('[3.13] eids should be given as a set', DeprecationWarning,
4c043afb104a fix failures introduced by recent refactoring
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7503
diff changeset
  1440
                 stacklevel=2)
4c043afb104a fix failures introduced by recent refactoring
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7503
diff changeset
  1441
            eids = frozenset(eids)
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1442
        eids = eids - op._container
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1443
        op._container |= eids
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1444
        data_by_etype_source = {} # values are ([list of eids],
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1445
                                  #             [list of extid],
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1446
                                  #             [list of entities])
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1447
        #
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1448
        # WARNING: the way this dictionary is populated is heavily optimized
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1449
        # and does not use setdefault on purpose. Unless a new release
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1450
        # of the Python interpreter advertises large perf improvements
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1451
        # in setdefault, this should not be changed without profiling.
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1452
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1453
        for eid in eids:
7543
570522300e22 [ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7514
diff changeset
  1454
            etype, sourceuri, extid, _ = self.type_and_source_from_eid(eid, session)
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1455
            # XXX should cache entity's cw_metainformation
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1456
            entity = session.entity_from_eid(eid, etype)
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1457
            try:
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1458
                data_by_etype_source[(etype, sourceuri)].append(entity)
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1459
            except KeyError:
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1460
                data_by_etype_source[(etype, sourceuri)] = [entity]
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1461
        for (etype, sourceuri), entities in data_by_etype_source.iteritems():
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1462
            if server.DEBUG & server.DBG_REPO:
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1463
                print 'DELETE entities', etype, [entity.eid for entity in entities]
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1464
            source = self.sources_by_uri[sourceuri]
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1465
            if source.should_call_hooks:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1466
                self.hm.call_hooks('before_delete_entity', session, entities=entities)
7898
405017c79b45 [ms, test] fix source deletion pb, detected by test since 7885:9454b7ef5ae4
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7895
diff changeset
  1467
            if session.deleted_in_transaction(source.eid):
405017c79b45 [ms, test] fix source deletion pb, detected by test since 7885:9454b7ef5ae4
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7895
diff changeset
  1468
                # source is being deleted, think to give scleanup argument
405017c79b45 [ms, test] fix source deletion pb, detected by test since 7885:9454b7ef5ae4
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7895
diff changeset
  1469
                self._delete_info_multi(session, entities, sourceuri, scleanup=source.eid)
405017c79b45 [ms, test] fix source deletion pb, detected by test since 7885:9454b7ef5ae4
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7895
diff changeset
  1470
            else:
405017c79b45 [ms, test] fix source deletion pb, detected by test since 7885:9454b7ef5ae4
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7895
diff changeset
  1471
                self._delete_info_multi(session, entities, sourceuri)
6889
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1472
            source.delete_entities(session, entities)
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1473
            if source.should_call_hooks:
37668bf302f5 improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 6801
diff changeset
  1474
                self.hm.call_hooks('after_delete_entity', session, entities=entities)
7501
2983dd24494a [repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7373
diff changeset
  1475
        # don't clear cache here, it is done in a hook on commit
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1476
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1477
    def glob_add_relation(self, session, subject, rtype, object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1478
        """add a relation to the repository"""
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1479
        self.glob_add_relations(session, {rtype: [(subject, object)]})
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1480
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1481
    def glob_add_relations(self, session, relations):
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1482
        """add several relations to the repository
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1483
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1484
        relations is a dictionary rtype: [(subj_eid, obj_eid), ...]
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1485
        """
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1486
        sources = {}
7513
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1487
        subjects_by_types = {}
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1488
        objects_by_types = {}
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1489
        activintegrity = session.is_hook_category_activated('activeintegrity')
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1490
        for rtype, eids_subj_obj in relations.iteritems():
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1491
            if server.DEBUG & server.DBG_REPO:
8548
eeabc752c32e [repo] fix debug code in DBG_REPO mode (closes #2469942)
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 8547
diff changeset
  1492
                for subjeid, objeid in eids_subj_obj:
7513
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1493
                    print 'ADD relation', subjeid, rtype, objeid
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1494
            for subjeid, objeid in eids_subj_obj:
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1495
                source = self.locate_relation_source(session, subjeid, rtype, objeid)
7238
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1496
                if source not in sources:
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1497
                    relations_by_rtype = {}
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1498
                    sources[source] = relations_by_rtype
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1499
                else:
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1500
                    relations_by_rtype = sources[source]
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1501
                if rtype in relations_by_rtype:
7513
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1502
                    relations_by_rtype[rtype].append((subjeid, objeid))
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1503
                else:
7513
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1504
                    relations_by_rtype[rtype] = [(subjeid, objeid)]
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1505
                if not activintegrity:
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1506
                    continue
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1507
                # take care to relation of cardinality '?1', as all eids will
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1508
                # be inserted later, we've remove duplicated eids since they
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1509
                # won't be catched by `del_existing_rel_if_needed`
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1510
                rdef = session.rtype_eids_rdef(rtype, subjeid, objeid)
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1511
                card = rdef.cardinality
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1512
                if card[0] in '?1':
8562
0d2fb4604265 [session] fix arguments default value and promote usage of security_enabled as session method. Closes #2481820
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8539
diff changeset
  1513
                    with session.security_enabled(read=False):
7513
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1514
                        session.execute('DELETE X %s Y WHERE X eid %%(x)s, '
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1515
                                        'NOT Y eid %%(y)s' % rtype,
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1516
                                        {'x': subjeid, 'y': objeid})
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1517
                    subjects = subjects_by_types.setdefault(rdef, {})
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1518
                    if subjeid in subjects:
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1519
                        del relations_by_rtype[rtype][subjects[subjeid]]
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1520
                        subjects[subjeid] = len(relations_by_rtype[rtype]) - 1
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1521
                        continue
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1522
                    subjects[subjeid] = len(relations_by_rtype[rtype]) - 1
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1523
                if card[1] in '?1':
8562
0d2fb4604265 [session] fix arguments default value and promote usage of security_enabled as session method. Closes #2481820
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 8539
diff changeset
  1524
                    with session.security_enabled(read=False):
7513
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1525
                        session.execute('DELETE X %s Y WHERE Y eid %%(y)s, '
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1526
                                        'NOT X eid %%(x)s' % rtype,
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1527
                                        {'x': subjeid, 'y': objeid})
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1528
                    objects = objects_by_types.setdefault(rdef, {})
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1529
                    if objeid in objects:
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1530
                        del relations_by_rtype[rtype][objects[objeid]]
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1531
                        objects[objeid] = len(relations_by_rtype[rtype])
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1532
                        continue
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1533
                    objects[objeid] = len(relations_by_rtype[rtype])
7238
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1534
        for source, relations_by_rtype in sources.iteritems():
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1535
            if source.should_call_hooks:
7238
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1536
                for rtype, source_relations in relations_by_rtype.iteritems():
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1537
                    self.hm.call_hooks('before_add_relation', session,
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1538
                                    rtype=rtype, eids_from_to=source_relations)
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1539
            for rtype, source_relations in relations_by_rtype.iteritems():
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1540
                source.add_relations(session, rtype, source_relations)
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1541
                rschema = self.schema.rschema(rtype)
7513
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1542
                for subjeid, objeid in source_relations:
8f4422391e5a [repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 7507
diff changeset
  1543
                    session.update_rel_cache_add(subjeid, rtype, objeid, rschema.symmetric)
7237
9f619715665b [server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7134
diff changeset
  1544
            if source.should_call_hooks:
7238
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1545
                for rtype, source_relations in relations_by_rtype.iteritems():
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1546
                    self.hm.call_hooks('after_add_relation', session,
576abb8c4626 fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7237
diff changeset
  1547
                                       rtype=rtype, eids_from_to=source_relations)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1548
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1549
    def glob_delete_relation(self, session, subject, rtype, object):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1550
        """delete a relation from the repository"""
2600
6cd6c5d11b45 [F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
  1551
        if server.DEBUG & server.DBG_REPO:
6cd6c5d11b45 [F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2596
diff changeset
  1552
            print 'DELETE relation', subject, rtype, object
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1553
        source = self.locate_relation_source(session, subject, rtype, object)
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1554
        if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1555
            self.hm.call_hooks('before_delete_relation', session,
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1556
                               eidfrom=subject, rtype=rtype, eidto=object)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1557
        source.delete_relation(session, subject, rtype, object)
2647
b0a2e779845c enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2641
diff changeset
  1558
        rschema = self.schema.rschema(rtype)
4467
0e73d299730a fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 4252
diff changeset
  1559
        session.update_rel_cache_del(subject, rtype, object, rschema.symmetric)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1560
        if source.should_call_hooks:
2835
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1561
            self.hm.call_hooks('after_delete_relation', session,
04034421b072 [hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2822
diff changeset
  1562
                               eidfrom=subject, rtype=rtype, eidto=object)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1563
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1564
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1565
    # pyro handling ###########################################################
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1566
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1567
    @property
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1568
    @cached
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1569
    def pyro_appid(self):
6061
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1570
        from logilab.common import pyro_ext as pyro
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1571
        config = self.config
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1572
        appid = '%s.%s' % pyro.ns_group_and_id(
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1573
            config['pyro-instance-id'] or config.appid,
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1574
            config['pyro-ns-group'])
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1575
        # ensure config['pyro-instance-id'] is a full qualified pyro name
7cb29eab1c9d [pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 5920
diff changeset
  1576
        config['pyro-instance-id'] = appid
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1577
        return appid
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1578
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1579
    def _use_pyrons(self):
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1580
        """return True if the pyro-ns-host is set to something else
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1581
        than NO_PYRONS, meaning we want to go through a pyro
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1582
        nameserver"""
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1583
        return self.config['pyro-ns-host'] != 'NO_PYRONS'
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1584
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1585
    def pyro_register(self, host=''):
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1586
        """register the repository as a pyro object"""
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1587
        from logilab.common import pyro_ext as pyro
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1588
        daemon = pyro.register_object(self, self.pyro_appid,
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1589
                                      daemonhost=self.config['pyro-host'],
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1590
                                      nshost=self.config['pyro-ns-host'],
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1591
                                      use_pyrons=self._use_pyrons())
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1592
        self.info('repository registered as a pyro object %s', self.pyro_appid)
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1593
        self.pyro_uri =  pyro.get_object_uri(self.pyro_appid)
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1594
        self.info('pyro uri is: %s', self.pyro_uri)
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1595
        self.pyro_registered = True
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1596
        # register a looping task to regularly ensure we're still registered
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1597
        # into the pyro name server
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1598
        if self._use_pyrons():
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1599
            self.looping_task(60*10, self._ensure_pyro_ns)
9007
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
  1600
        pyro_sessions = self._pyro_sessions
8953
041804bc48e9 [server] add a looping task to free unused cnxsets capture by dead pyro
Julien Cristau <julien.cristau@logilab.fr>
parents: 8890
diff changeset
  1601
        # install hacky function to free cnxset
9007
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
  1602
        def handleConnection(conn, tcpserver, sessions=pyro_sessions):
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
  1603
            sessions[threading.currentThread()] = None
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
  1604
            return tcpserver.getAdapter().__class__.handleConnection(tcpserver.getAdapter(), conn, tcpserver)
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
  1605
        daemon.getAdapter().handleConnection = handleConnection
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
  1606
        def removeConnection(conn, sessions=pyro_sessions):
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
  1607
            daemon.__class__.removeConnection(daemon, conn)
9008
e0aa7cf8abf8 repository: make tests pass again
Julien Cristau <julien.cristau@logilab.fr>
parents: 9007
diff changeset
  1608
            session = sessions.pop(threading.currentThread(), None)
9007
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
  1609
            if session is None:
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
  1610
                # client was not yet connected to the repo
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
  1611
                return
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
  1612
            if not session.closed:
9132
b6bcf99504c9 [server/repository] Go through the repo to close pyro sessions
Julien Cristau <julien.cristau@logilab.fr>
parents: 9008
diff changeset
  1613
                self.close(session.id)
9007
e27337dfec8c repository: monkey patch pyro connection handling to detect clients going away
Julien Cristau <julien.cristau@logilab.fr>
parents: 8954
diff changeset
  1614
        daemon.removeConnection = removeConnection
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1615
        return daemon
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1616
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1617
    def _ensure_pyro_ns(self):
7134
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1618
        if not self._use_pyrons():
01544b6d98fa allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7085
diff changeset
  1619
            return
6801
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1620
        from logilab.common import pyro_ext as pyro
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1621
        pyro.ns_reregister(self.pyro_appid, nshost=self.config['pyro-ns-host'])
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1622
        self.info('repository re-registered as a pyro object %s',
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1623
                  self.pyro_appid)
33952695295b [repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6722
diff changeset
  1624
1228
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1625
    # multi-sources planner helpers ###########################################
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1626
1228
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1627
    @cached
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1628
    def rel_type_sources(self, rtype):
9302
af36f459d2f0 [server] deprecate the old multi-source system
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9272
diff changeset
  1629
        warn('[3.18] old multi-source system will go away in the next version',
af36f459d2f0 [server] deprecate the old multi-source system
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9272
diff changeset
  1630
             DeprecationWarning)
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1631
        return tuple([source for source in self.sources
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1632
                      if source.support_relation(rtype)
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1633
                      or rtype in source.dont_cross_relations])
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1634
1228
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1635
    @cached
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1636
    def can_cross_relation(self, rtype):
9302
af36f459d2f0 [server] deprecate the old multi-source system
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9272
diff changeset
  1637
        warn('[3.18] old multi-source system will go away in the next version',
af36f459d2f0 [server] deprecate the old multi-source system
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9272
diff changeset
  1638
             DeprecationWarning)
6128
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1639
        return tuple([source for source in self.sources
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1640
                      if source.support_relation(rtype)
fbb8398f80dc cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 6126
diff changeset
  1641
                      and rtype in source.cross_relations])
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1642
1228
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1643
    @cached
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1644
    def is_multi_sources_relation(self, rtype):
9302
af36f459d2f0 [server] deprecate the old multi-source system
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9272
diff changeset
  1645
        warn('[3.18] old multi-source system will go away in the next version',
af36f459d2f0 [server] deprecate the old multi-source system
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents: 9272
diff changeset
  1646
             DeprecationWarning)
1228
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1647
        return any(source for source in self.sources
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1648
                   if not source is self.system_source
91ae10ffb611 * refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents: 1217
diff changeset
  1649
                   and source.support_relation(rtype))
1482
93c613913912 delete-trailing-whitespaces
sylvain.thenault@logilab.fr
parents: 1398
diff changeset
  1650
7083
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7064
diff changeset
  1651
    # these are overridden by set_log_methods below
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7064
diff changeset
  1652
    # only defining here to prevent pylint from complaining
b8e35cde46e9 help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents: 7064
diff changeset
  1653
    info = warning = error = critical = exception = debug = lambda msg,*a,**kw: None
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1654
8268
c9babe49c1c1 [repository] implement a generic way to call repo-side services; closes #2203418
Florent Cayré <florent.cayre@logilab.fr>
parents: 8239
diff changeset
  1655
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1656
def pyro_unregister(config):
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1657
    """unregister the repository from the pyro name server"""
2665
0c6281487f90 [pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2650
diff changeset
  1658
    from logilab.common.pyro_ext import ns_unregister
3539
f3b14d052798 [pyro] merge pyro-id / pyro-instance-id options, put all pyro options in the same section of the configuration file
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 3250
diff changeset
  1659
    appid = config['pyro-instance-id'] or config.appid
2665
0c6281487f90 [pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents: 2650
diff changeset
  1660
    ns_unregister(appid, config['pyro-ns-group'], config['pyro-ns-host'])
0
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1661
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1662
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1663
from logging import getLogger
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1664
from cubicweb import set_log_methods
b97547f5f1fa Showtime !
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
diff changeset
  1665
set_log_methods(Repository, getLogger('cubicweb.repository'))