author | Aurelien Campeas <aurelien.campeas@logilab.fr> |
Fri, 20 May 2011 19:27:00 +0200 | |
changeset 7415 | 777c47312bc7 |
parent 7399 | 972ed1843bd8 |
child 7514 | 32081892850e |
permissions | -rw-r--r-- |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1 |
# copyright 2003-2011 LOGILAB S.A. (Paris, FRANCE), all rights reserved. |
5421
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
2 |
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
3 |
# |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
4 |
# This file is part of CubicWeb. |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
5 |
# |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
6 |
# CubicWeb is free software: you can redistribute it and/or modify it under the |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
7 |
# terms of the GNU Lesser General Public License as published by the Free |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
8 |
# Software Foundation, either version 2.1 of the License, or (at your option) |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
9 |
# any later version. |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
10 |
# |
5424
8ecbcbff9777
replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5421
diff
changeset
|
11 |
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT |
5421
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
12 |
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
13 |
# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
14 |
# details. |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
15 |
# |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
16 |
# You should have received a copy of the GNU Lesser General Public License along |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
17 |
# with CubicWeb. If not, see <http://www.gnu.org/licenses/>. |
0 | 18 |
"""Defines the central class for the CubicWeb RQL server: the repository. |
19 |
||
20 |
The repository is an abstraction allowing execution of rql queries against |
|
21 |
data sources. Most of the work is actually done in helper classes. The |
|
22 |
repository mainly: |
|
23 |
||
24 |
* brings these classes all together to provide a single access |
|
2476
1294a6bdf3bf
application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2473
diff
changeset
|
25 |
point to a cubicweb instance. |
0 | 26 |
* handles session management |
27 |
* provides method for pyro registration, to call if pyro is enabled |
|
5587
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
28 |
""" |
0 | 29 |
|
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
30 |
from __future__ import with_statement |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
31 |
|
0 | 32 |
__docformat__ = "restructuredtext en" |
33 |
||
34 |
import sys |
|
5587
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
35 |
import threading |
0 | 36 |
import Queue |
6390
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
37 |
from itertools import chain |
4719
aaed3f813ef8
kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4714
diff
changeset
|
38 |
from os.path import join |
1016
26387b836099
use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents:
636
diff
changeset
|
39 |
from datetime import datetime |
0 | 40 |
from time import time, localtime, strftime |
41 |
||
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
42 |
from logilab.common.decorators import cached, clear_cache |
3757 | 43 |
from logilab.common.compat import any |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
44 |
from logilab.common import flatten |
0 | 45 |
|
46 |
from yams import BadSchemaDefinition |
|
5030
5238d9a8dfee
[form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4984
diff
changeset
|
47 |
from yams.schema import role_name |
0 | 48 |
from rql import RQLSyntaxError |
6390
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
49 |
from rql.utils import rqlvar_maker |
0 | 50 |
|
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
51 |
from cubicweb import (CW_SOFTWARE_ROOT, CW_MIGRATION_MAP, QueryError, |
2709
6ee816eb9f25
[hooksmanager] hooks should be reseted after vreg reload
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2708
diff
changeset
|
52 |
UnknownEid, AuthenticationError, ExecutionError, |
3042
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
53 |
ETypeNotSupportedBySources, MultiSourcesError, |
0 | 54 |
BadConnectionId, Unauthorized, ValidationError, |
6211
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
55 |
RepositoryError, UniqueTogetherError, typed_eid, onevent) |
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
56 |
from cubicweb import cwvreg, schema, server |
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
57 |
from cubicweb.server import utils, hook, pool, querier, sources |
5079
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
58 |
from cubicweb.server.session import Session, InternalSession, InternalManager, \ |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
59 |
security_enabled |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
60 |
from cubicweb.server.ssplanner import EditedEntity |
0 | 61 |
|
7373
e5e6ef56cfb5
[repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7305
diff
changeset
|
62 |
NO_CACHE_RELATIONS = set( [('require_permission', 'object'), |
e5e6ef56cfb5
[repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7305
diff
changeset
|
63 |
('owned_by', 'object'), |
e5e6ef56cfb5
[repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7305
diff
changeset
|
64 |
('created_by', 'object'), |
e5e6ef56cfb5
[repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7305
diff
changeset
|
65 |
('cw_source', 'object'), |
e5e6ef56cfb5
[repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7305
diff
changeset
|
66 |
]) |
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
67 |
|
6467
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
68 |
def prefill_entity_caches(entity, relations): |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
69 |
session = entity._cw |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
70 |
# prefill entity relation caches |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
71 |
for rschema in entity.e_schema.subject_relations(): |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
72 |
rtype = str(rschema) |
7373
e5e6ef56cfb5
[repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7305
diff
changeset
|
73 |
if rtype in schema.VIRTUAL_RTYPES or (rtype, 'subject') in NO_CACHE_RELATIONS: |
6467
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
74 |
continue |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
75 |
if rschema.final: |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
76 |
entity.cw_attr_cache.setdefault(rtype, None) |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
77 |
else: |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
78 |
entity.cw_set_relation_cache(rtype, 'subject', |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
79 |
session.empty_rset()) |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
80 |
for rschema in entity.e_schema.object_relations(): |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
81 |
rtype = str(rschema) |
7373
e5e6ef56cfb5
[repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7305
diff
changeset
|
82 |
if rtype in schema.VIRTUAL_RTYPES or (rtype, 'object') in NO_CACHE_RELATIONS: |
6467
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
83 |
continue |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
84 |
entity.cw_set_relation_cache(rtype, 'object', session.empty_rset()) |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
85 |
# set inlined relation cache before call to after_add_entity |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
86 |
for attr, value in relations: |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
87 |
session.update_rel_cache_add(entity.eid, attr, value) |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
88 |
del_existing_rel_if_needed(session, entity.eid, attr, value) |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
89 |
|
0 | 90 |
def del_existing_rel_if_needed(session, eidfrom, rtype, eidto): |
91 |
"""delete existing relation when adding a new one if card is 1 or ? |
|
92 |
||
93 |
have to be done once the new relation has been inserted to avoid having |
|
94 |
an entity without a relation for some time |
|
95 |
||
96 |
this kind of behaviour has to be done in the repository so we don't have |
|
97 |
hooks order hazardness |
|
98 |
""" |
|
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
99 |
# skip that for internal session or if integrity explicitly disabled |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
100 |
# |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
101 |
# XXX we should imo rely on the orm to first fetch existing entity if any |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
102 |
# then delete it. |
4756
a2c73b272c9b
do not check for existant entity when relation of single card is added but integrity hooks are deactivated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4719
diff
changeset
|
103 |
if session.is_internal_session \ |
5090
8c39d2bf58fd
[repo creation] removing existing entities of 'single' cardinality relatino should be considered as 'activeintegrity' hook. Also don't disable that category during repo creation to avoid pb such as two default workflows for one entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5084
diff
changeset
|
104 |
or not session.is_hook_category_activated('activeintegrity'): |
4687
082e66184f71
[integrity] skip auto-delete for internal sessions (but *not* for super session, see 4642:921737d2e3a8)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4686
diff
changeset
|
105 |
return |
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
106 |
card = session.schema_rproperty(rtype, eidfrom, eidto, 'cardinality') |
0 | 107 |
# one may be tented to check for neweids but this may cause more than one |
108 |
# relation even with '1?' cardinality if thoses relations are added in the |
|
109 |
# same transaction where the entity is being created. This never occurs from |
|
110 |
# the web interface but may occurs during test or dbapi connection (though |
|
111 |
# not expected for this). So: don't do it, we pretend to ensure repository |
|
112 |
# consistency. |
|
4673
6f8b925a29f4
auto-delete relation with single cardinality using execute, not unsafe_execute, so permissions are correctly checked
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4643
diff
changeset
|
113 |
# |
6189 | 114 |
# notes: |
115 |
# * inlined relations will be implicitly deleted for the subject entity |
|
116 |
# * we don't want read permissions to be applied but we want delete |
|
117 |
# permission to be checked |
|
118 |
if card[0] in '1?' and not session.repo.schema.rschema(rtype).inlined: |
|
119 |
with security_enabled(session, read=False): |
|
120 |
session.execute('DELETE X %s Y WHERE X eid %%(x)s, ' |
|
121 |
'NOT Y eid %%(y)s' % rtype, |
|
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
122 |
{'x': eidfrom, 'y': eidto}) |
0 | 123 |
if card[1] in '1?': |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
124 |
with security_enabled(session, read=False): |
5711
ea2538a3c622
[repo] fix typo
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5641
diff
changeset
|
125 |
session.execute('DELETE X %s Y WHERE Y eid %%(y)s, ' |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
126 |
'NOT X eid %%(x)s' % rtype, |
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
127 |
{'x': eidfrom, 'y': eidto}) |
1482 | 128 |
|
3694 | 129 |
|
0 | 130 |
class Repository(object): |
131 |
"""a repository provides access to a set of persistent storages for |
|
132 |
entities and relations |
|
133 |
||
134 |
XXX protect pyro access |
|
135 |
""" |
|
1482 | 136 |
|
5442
3ed8afbbdf70
[webconfig] refactor/cleanup debug mode management on startup: simply use config.debugmode instead of debug argument everywhere...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
137 |
def __init__(self, config, vreg=None): |
0 | 138 |
self.config = config |
139 |
if vreg is None: |
|
5442
3ed8afbbdf70
[webconfig] refactor/cleanup debug mode management on startup: simply use config.debugmode instead of debug argument everywhere...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
140 |
vreg = cwvreg.CubicWebVRegistry(config) |
0 | 141 |
self.vreg = vreg |
142 |
self.pyro_registered = False |
|
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
143 |
self.pyro_uri = None |
0 | 144 |
self.info('starting repository from %s', self.config.apphome) |
145 |
# dictionary of opened sessions |
|
146 |
self._sessions = {} |
|
147 |
# list of functions to be called at regular interval |
|
148 |
self._looping_tasks = [] |
|
149 |
# list of running threads |
|
150 |
self._running_threads = [] |
|
151 |
# initial schema, should be build or replaced latter |
|
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
152 |
self.schema = schema.CubicWebSchema(config.appid) |
3240
8604a15995d1
refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3042
diff
changeset
|
153 |
self.vreg.schema = self.schema # until actual schema is loaded... |
5906
d40ced753291
[repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5813
diff
changeset
|
154 |
# shutdown flag |
d40ced753291
[repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5813
diff
changeset
|
155 |
self.shutting_down = False |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
156 |
# sources (additional sources info in the system database) |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
157 |
self.system_source = self.get_source('native', 'system', |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
158 |
config.sources()['system']) |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
159 |
self.sources = [self.system_source] |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
160 |
self.sources_by_uri = {'system': self.system_source} |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
161 |
# querier helper, need to be created after sources initialization |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
162 |
self.querier = querier.QuerierHelper(self, self.schema) |
7399
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
163 |
# cache eid -> (type, source, extid) |
0 | 164 |
self._type_source_cache = {} |
165 |
# cache (extid, source uri) -> eid |
|
166 |
self._extid_cache = {} |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
167 |
# open some connections set |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
168 |
if config.init_cnxset_pool: |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
169 |
self.init_cnxset_pool() |
5079
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
170 |
@onevent('after-registry-reload', self) |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
171 |
def fix_user_classes(self): |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
172 |
usercls = self.vreg['etypes'].etype_class('CWUser') |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
173 |
for session in self._sessions.values(): |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
174 |
if not isinstance(session.user, InternalManager): |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
175 |
session.user.__class__ = usercls |
2959
daabb9bc5233
make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2929
diff
changeset
|
176 |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
177 |
def init_cnxset_pool(self): |
2959
daabb9bc5233
make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2929
diff
changeset
|
178 |
config = self.config |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
179 |
self._cnxsets_pool = Queue.Queue() |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
180 |
self._cnxsets_pool.put_nowait(pool.ConnectionsSet(self.sources)) |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
181 |
if config.quick_start: |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
182 |
# quick start, usually only to get a minimal repository to get cubes |
5084
d8f491cb046c
[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5079
diff
changeset
|
183 |
# information (eg dump/restore/...) |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
184 |
config._cubes = () |
5084
d8f491cb046c
[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5079
diff
changeset
|
185 |
# only load hooks and entity classes in the registry |
7305
1f97c05cfbab
[repository quick start] should modify class attribute to get this actually working (used by a classmethod)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7238
diff
changeset
|
186 |
config.__class__.cube_appobject_path = set(('hooks', 'entities')) |
1f97c05cfbab
[repository quick start] should modify class attribute to get this actually working (used by a classmethod)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7238
diff
changeset
|
187 |
config.__class__.cubicweb_appobject_path = set(('hooks', 'entities')) |
5084
d8f491cb046c
[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5079
diff
changeset
|
188 |
self.set_schema(config.load_schema()) |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
189 |
config['connections-pool-size'] = 1 |
5084
d8f491cb046c
[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5079
diff
changeset
|
190 |
# will be reinitialized later from cubes found in the database |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
191 |
config._cubes = None |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
192 |
elif config.creating: |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
193 |
# repository creation |
0 | 194 |
config.bootstrap_cubes() |
2968
0e3460341023
somewhat painful backport of 3.5 branch, should mostly be ok
Sylvain Thénault <sylvain.thenault@logilab.fr>
diff
changeset
|
195 |
self.set_schema(config.load_schema(), resetvreg=False) |
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
196 |
# need to load the Any and CWUser entity types |
0 | 197 |
etdirectory = join(CW_SOFTWARE_ROOT, 'entities') |
1317 | 198 |
self.vreg.init_registration([etdirectory]) |
3409
c684606a95e0
[bootstrap] metadata hooks need to be registered at bootstrap time for db creation
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
3381
diff
changeset
|
199 |
for modname in ('__init__', 'authobjs', 'wfobjs'): |
c684606a95e0
[bootstrap] metadata hooks need to be registered at bootstrap time for db creation
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
3381
diff
changeset
|
200 |
self.vreg.load_file(join(etdirectory, '%s.py' % modname), |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
201 |
'cubicweb.entities.%s' % modname) |
5447
f29dafba250a
[vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5442
diff
changeset
|
202 |
hooksdirectory = join(CW_SOFTWARE_ROOT, 'hooks') |
f29dafba250a
[vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5442
diff
changeset
|
203 |
self.vreg.load_file(join(hooksdirectory, 'metadata.py'), |
f29dafba250a
[vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5442
diff
changeset
|
204 |
'cubicweb.hooks.metadata') |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
205 |
elif config.read_instance_schema: |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
206 |
# normal start: load the instance schema from the database |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
207 |
self.fill_schema() |
0 | 208 |
else: |
209 |
# test start: use the file system schema (quicker) |
|
2476
1294a6bdf3bf
application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2473
diff
changeset
|
210 |
self.warning("set fs instance'schema") |
0 | 211 |
config.bootstrap_cubes() |
2961 | 212 |
self.set_schema(config.load_schema()) |
0 | 213 |
if not config.creating: |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
214 |
self.init_sources_from_database() |
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
215 |
if 'CWProperty' in self.schema: |
0 | 216 |
self.vreg.init_properties(self.properties()) |
217 |
else: |
|
6128 | 218 |
# call init_creating so that for instance native source can |
219 |
# configurate tsearch according to postgres version |
|
0 | 220 |
for source in self.sources: |
221 |
source.init_creating() |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
222 |
# close initialization connetions set and reopen fresh ones for proper |
0 | 223 |
# initialization now that we know cubes |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
224 |
self._get_cnxset().close(True) |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
225 |
# list of available_cnxsets (we can't iterate on Queue instance) |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
226 |
self.cnxsets = [] |
0 | 227 |
for i in xrange(config['connections-pool-size']): |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
228 |
self.cnxsets.append(pool.ConnectionsSet(self.sources)) |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
229 |
self._cnxsets_pool.put_nowait(self.cnxsets[-1]) |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
230 |
if config.quick_start: |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
231 |
config.init_cubes(self.get_cubes()) |
5093
8d073d2e089d
[optimization] improve massive write performance by optimizing hooks selection
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5090
diff
changeset
|
232 |
self.hm = hook.HooksManager(self.vreg) |
1482 | 233 |
|
0 | 234 |
# internals ############################################################### |
235 |
||
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
236 |
def init_sources_from_database(self): |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
237 |
self.sources_by_eid = {} |
6664
7b6acf314a7a
[repo] don't attempt to read sources from database on quick start. Fix crash after restoring a pre 3.10 database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6629
diff
changeset
|
238 |
if self.config.quick_start \ |
7b6acf314a7a
[repo] don't attempt to read sources from database on quick start. Fix crash after restoring a pre 3.10 database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6629
diff
changeset
|
239 |
or not 'CWSource' in self.schema: # # 3.10 migration |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
240 |
return |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
241 |
session = self.internal_session() |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
242 |
try: |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
243 |
# FIXME: sources should be ordered (add_entity priority) |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
244 |
for sourceent in session.execute( |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
245 |
'Any S, SN, SA, SC WHERE S is_instance_of CWSource, ' |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
246 |
'S name SN, S type SA, S config SC').entities(): |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
247 |
if sourceent.name == 'system': |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
248 |
self.system_source.eid = sourceent.eid |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
249 |
self.sources_by_eid[sourceent.eid] = self.system_source |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
250 |
self.system_source.init(True, sourceent) |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
251 |
continue |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
252 |
self.add_source(sourceent, add_to_cnxsets=False) |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
253 |
finally: |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
254 |
session.close() |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
255 |
|
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
256 |
def _clear_planning_caches(self): |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
257 |
for cache in ('source_defs', 'is_multi_sources_relation', |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
258 |
'can_cross_relation', 'rel_type_sources'): |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
259 |
clear_cache(self, cache) |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
260 |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
261 |
def add_source(self, sourceent, add_to_cnxsets=True): |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
262 |
source = self.get_source(sourceent.type, sourceent.name, |
6945
28bf94d062a9
[sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6941
diff
changeset
|
263 |
sourceent.host_config, sourceent.eid) |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
264 |
self.sources_by_eid[sourceent.eid] = source |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
265 |
self.sources_by_uri[sourceent.name] = source |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
266 |
if self.config.source_enabled(source): |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
267 |
# call source's init method to complete their initialisation if |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
268 |
# needed (for instance looking for persistent configuration using an |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
269 |
# internal session, which is not possible until connections sets have been |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
270 |
# initialized) |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
271 |
source.init(True, sourceent) |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
272 |
if not source.copy_based_source: |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
273 |
self.sources.append(source) |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
274 |
self.querier.set_planner() |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
275 |
if add_to_cnxsets: |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
276 |
for cnxset in self.cnxsets: |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
277 |
cnxset.add_source(source) |
6724
24bf6f181d0e
[pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
278 |
else: |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
279 |
source.init(False, sourceent) |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
280 |
self._clear_planning_caches() |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
281 |
|
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
282 |
def remove_source(self, uri): |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
283 |
source = self.sources_by_uri.pop(uri) |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
284 |
del self.sources_by_eid[source.eid] |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
285 |
if self.config.source_enabled(source) and not source.copy_based_source: |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
286 |
self.sources.remove(source) |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
287 |
self.querier.set_planner() |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
288 |
for cnxset in self.cnxsets: |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
289 |
cnxset.remove_source(source) |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
290 |
self._clear_planning_caches() |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
291 |
|
6945
28bf94d062a9
[sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6941
diff
changeset
|
292 |
def get_source(self, type, uri, source_config, eid=None): |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
293 |
# set uri and type in source config so it's available through |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
294 |
# source_defs() |
0 | 295 |
source_config['uri'] = uri |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
296 |
source_config['type'] = type |
6945
28bf94d062a9
[sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6941
diff
changeset
|
297 |
return sources.get_source(type, source_config, self, eid) |
1482 | 298 |
|
2963
12ad88615a12
test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2961
diff
changeset
|
299 |
def set_schema(self, schema, resetvreg=True, rebuildinfered=True): |
12ad88615a12
test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2961
diff
changeset
|
300 |
if rebuildinfered: |
12ad88615a12
test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2961
diff
changeset
|
301 |
schema.rebuild_infered_relations() |
0 | 302 |
self.info('set schema %s %#x', schema.name, id(schema)) |
3240
8604a15995d1
refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3042
diff
changeset
|
303 |
if resetvreg: |
2881
d1a5b77e42bc
fix set_schema/init_cubes order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2880
diff
changeset
|
304 |
if self.config._cubes is None: |
d1a5b77e42bc
fix set_schema/init_cubes order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2880
diff
changeset
|
305 |
self.config.init_cubes(self.get_cubes()) |
5447
f29dafba250a
[vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5442
diff
changeset
|
306 |
# trigger full reload of all appobjects |
3240
8604a15995d1
refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3042
diff
changeset
|
307 |
self.vreg.set_schema(schema) |
8604a15995d1
refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3042
diff
changeset
|
308 |
else: |
8604a15995d1
refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3042
diff
changeset
|
309 |
self.vreg._set_schema(schema) |
0 | 310 |
self.querier.set_schema(schema) |
6126
aca6a2c357fd
[repository] enabled sources refactoring: to avoid error, we should always have all known source in repo.sources_by_uri and only enabled ones in repo.sources, so we still have access to the definition of temporarily disabled sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6061
diff
changeset
|
311 |
# don't use self.sources, we may want to give schema even to disabled |
aca6a2c357fd
[repository] enabled sources refactoring: to avoid error, we should always have all known source in repo.sources_by_uri and only enabled ones in repo.sources, so we still have access to the definition of temporarily disabled sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6061
diff
changeset
|
312 |
# sources |
aca6a2c357fd
[repository] enabled sources refactoring: to avoid error, we should always have all known source in repo.sources_by_uri and only enabled ones in repo.sources, so we still have access to the definition of temporarily disabled sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6061
diff
changeset
|
313 |
for source in self.sources_by_uri.values(): |
0 | 314 |
source.set_schema(schema) |
315 |
self.schema = schema |
|
316 |
||
317 |
def fill_schema(self): |
|
318 |
"""lod schema from the repository""" |
|
319 |
from cubicweb.server.schemaserial import deserialize_schema |
|
320 |
self.info('loading schema from the repository') |
|
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
321 |
appschema = schema.CubicWebSchema(self.config.appid) |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
322 |
self.set_schema(self.config.load_bootstrap_schema(), resetvreg=False) |
0 | 323 |
self.debug('deserializing db schema into %s %#x', appschema.name, id(appschema)) |
324 |
session = self.internal_session() |
|
325 |
try: |
|
326 |
try: |
|
327 |
deserialize_schema(appschema, session) |
|
328 |
except BadSchemaDefinition: |
|
329 |
raise |
|
330 |
except Exception, ex: |
|
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
331 |
import traceback |
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
332 |
traceback.print_exc() |
1482 | 333 |
raise Exception('Is the database initialised ? (cause: %s)' % |
0 | 334 |
(ex.args and ex.args[0].strip() or 'unknown')), \ |
335 |
None, sys.exc_info()[-1] |
|
336 |
finally: |
|
337 |
session.close() |
|
338 |
self.set_schema(appschema) |
|
1482 | 339 |
|
0 | 340 |
def start_looping_tasks(self): |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
341 |
if not (self.config.creating or self.config.repairing |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
342 |
or self.config.quick_start): |
4958
665eacdd8c50
[repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4951
diff
changeset
|
343 |
# call instance level initialisation hooks |
665eacdd8c50
[repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4951
diff
changeset
|
344 |
self.hm.call_hooks('server_startup', repo=self) |
665eacdd8c50
[repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4951
diff
changeset
|
345 |
# register a task to cleanup expired session |
6012
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
346 |
self.cleanup_session_time = self.config['cleanup-session-time'] or 60 * 60 * 24 |
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
347 |
assert self.cleanup_session_time > 0 |
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
348 |
cleanup_session_interval = min(60*60, self.cleanup_session_time / 3) |
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
349 |
self.looping_task(cleanup_session_interval, self.clean_sessions) |
0 | 350 |
assert isinstance(self._looping_tasks, list), 'already started' |
2708
60d728bdcba5
allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2667
diff
changeset
|
351 |
for i, (interval, func, args) in enumerate(self._looping_tasks): |
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
352 |
self._looping_tasks[i] = task = utils.LoopTask(interval, func, args) |
0 | 353 |
self.info('starting task %s with interval %.2fs', task.name, |
354 |
interval) |
|
355 |
task.start() |
|
356 |
# ensure no tasks will be further added |
|
357 |
self._looping_tasks = tuple(self._looping_tasks) |
|
358 |
||
2708
60d728bdcba5
allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2667
diff
changeset
|
359 |
def looping_task(self, interval, func, *args): |
0 | 360 |
"""register a function to be called every `interval` seconds. |
1482 | 361 |
|
0 | 362 |
looping tasks can only be registered during repository initialization, |
363 |
once done this method will fail. |
|
364 |
""" |
|
365 |
try: |
|
2708
60d728bdcba5
allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2667
diff
changeset
|
366 |
self._looping_tasks.append( (interval, func, args) ) |
0 | 367 |
except AttributeError: |
368 |
raise RuntimeError("can't add looping task once the repository is started") |
|
369 |
||
370 |
def threaded_task(self, func): |
|
371 |
"""start function in a separated thread""" |
|
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
372 |
t = utils.RepoThread(func, self._running_threads) |
0 | 373 |
t.start() |
1482 | 374 |
|
0 | 375 |
#@locked |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
376 |
def _get_cnxset(self): |
0 | 377 |
try: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
378 |
return self._cnxsets_pool.get(True, timeout=5) |
0 | 379 |
except Queue.Empty: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
380 |
raise Exception('no connections set available after 5 secs, probably either a ' |
4706
6035e96b64dd
added stats for munin collecting #615844 - from 027bbff3659f
arthur
parents:
4689
diff
changeset
|
381 |
'bug in code (too many uncommited/rollbacked ' |
6035e96b64dd
added stats for munin collecting #615844 - from 027bbff3659f
arthur
parents:
4689
diff
changeset
|
382 |
'connections) or too much load on the server (in ' |
0 | 383 |
'which case you can try to set a bigger ' |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
384 |
'connections pool size)') |
1482 | 385 |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
386 |
def _free_cnxset(self, cnxset): |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
387 |
self._cnxsets_pool.put_nowait(cnxset) |
0 | 388 |
|
389 |
def pinfo(self): |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
390 |
# XXX: session.cnxset is accessed from a local storage, would be interesting |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
391 |
# to see if there is a cnxset set in any thread specific data) |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
392 |
return '%s: %s (%s)' % (self._cnxsets_pool.qsize(), |
0 | 393 |
','.join(session.user.login for session in self._sessions.values() |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
394 |
if session.cnxset), |
0 | 395 |
threading.currentThread()) |
396 |
def shutdown(self): |
|
397 |
"""called on server stop event to properly close opened sessions and |
|
398 |
connections |
|
399 |
""" |
|
5906
d40ced753291
[repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5813
diff
changeset
|
400 |
assert not self.shutting_down, 'already shutting down' |
d40ced753291
[repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5813
diff
changeset
|
401 |
self.shutting_down = True |
5749
b4393b681f7a
[repo] on repository shutdown, we've to close the new eid creation connection
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5747
diff
changeset
|
402 |
self.system_source.shutdown() |
0 | 403 |
if isinstance(self._looping_tasks, tuple): # if tasks have been started |
404 |
for looptask in self._looping_tasks: |
|
405 |
self.info('canceling task %s...', looptask.name) |
|
406 |
looptask.cancel() |
|
407 |
looptask.join() |
|
408 |
self.info('task %s finished', looptask.name) |
|
409 |
for thread in self._running_threads: |
|
5376
2c3f14bc2590
[python2.6] don't add a name property on Thread
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5292
diff
changeset
|
410 |
self.info('waiting thread %s...', thread.getName()) |
0 | 411 |
thread.join() |
5376
2c3f14bc2590
[python2.6] don't add a name property on Thread
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5292
diff
changeset
|
412 |
self.info('thread %s finished', thread.getName()) |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
413 |
if not (self.config.creating or self.config.repairing |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
414 |
or self.config.quick_start): |
4946
2654cd5c87e0
[repo] don't call server_shutdown hooks when creating/repairing instance to be consistence with server_startup hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4862
diff
changeset
|
415 |
self.hm.call_hooks('server_shutdown', repo=self) |
0 | 416 |
self.close_sessions() |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
417 |
while not self._cnxsets_pool.empty(): |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
418 |
cnxset = self._cnxsets_pool.get_nowait() |
0 | 419 |
try: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
420 |
cnxset.close(True) |
0 | 421 |
except: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
422 |
self.exception('error while closing %s' % cnxset) |
0 | 423 |
continue |
424 |
if self.pyro_registered: |
|
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
425 |
if self._use_pyrons(): |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
426 |
pyro_unregister(self.config) |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
427 |
self.pyro_uri = None |
0 | 428 |
hits, misses = self.querier.cache_hit, self.querier.cache_miss |
429 |
try: |
|
5747
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
430 |
self.info('rql st cache hit/miss: %s/%s (%s%% hits)', hits, misses, |
0 | 431 |
(hits * 100) / (hits + misses)) |
432 |
hits, misses = self.system_source.cache_hit, self.system_source.cache_miss |
|
433 |
self.info('sql cache hit/miss: %s/%s (%s%% hits)', hits, misses, |
|
434 |
(hits * 100) / (hits + misses)) |
|
435 |
nocache = self.system_source.no_cache |
|
436 |
self.info('sql cache usage: %s/%s (%s%%)', hits+ misses, nocache, |
|
437 |
((hits + misses) * 100) / (hits + misses + nocache)) |
|
438 |
except ZeroDivisionError: |
|
439 |
pass |
|
1482 | 440 |
|
6849
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
441 |
def check_auth_info(self, session, login, authinfo): |
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
442 |
"""validate authentication, raise AuthenticationError on failure, return |
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
443 |
associated CWUser's eid on success. |
0 | 444 |
""" |
445 |
for source in self.sources: |
|
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
446 |
if source.support_entity('CWUser'): |
0 | 447 |
try: |
6849
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
448 |
return source.authenticate(session, login, **authinfo) |
0 | 449 |
except AuthenticationError: |
450 |
continue |
|
451 |
else: |
|
452 |
raise AuthenticationError('authentication failed with all sources') |
|
6849
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
453 |
|
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
454 |
def authenticate_user(self, session, login, **authinfo): |
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
455 |
"""validate login / password, raise AuthenticationError on failure |
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
456 |
return associated CWUser instance on success |
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
457 |
""" |
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
458 |
eid = self.check_auth_info(session, login, authinfo) |
2268
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
459 |
cwuser = self._build_user(session, eid) |
0 | 460 |
if self.config.consider_user_state and \ |
5556
9ab2b4c74baf
[entity] introduce a new 'adapters' registry
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5447
diff
changeset
|
461 |
not cwuser.cw_adapt_to('IWorkflowable').state in cwuser.AUTHENTICABLE_STATES: |
0 | 462 |
raise AuthenticationError('user is not in authenticable state') |
2268
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
463 |
return cwuser |
0 | 464 |
|
465 |
def _build_user(self, session, eid): |
|
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
466 |
"""return a CWUser entity for user with the given eid""" |
2650
18aec79ec3a3
R [vreg] important refactoring of the vregistry, moving behaviour to end dictionnary (and so leaving room for more flexibility ; keep bw compat ; update api usage in cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2647
diff
changeset
|
467 |
cls = self.vreg['etypes'].etype_class('CWUser') |
0 | 468 |
rql = cls.fetch_rql(session.user, ['X eid %(x)s']) |
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
469 |
rset = session.execute(rql, {'x': eid}) |
0 | 470 |
assert len(rset) == 1, rset |
2268
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
471 |
cwuser = rset.get_entity(0, 0) |
6491
ee9a10b6620e
pylint option update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6467
diff
changeset
|
472 |
# pylint: disable=W0104 |
2268
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
473 |
# prefetch / cache cwuser's groups and properties. This is especially |
0 | 474 |
# useful for internal sessions to avoid security insertions |
2268
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
475 |
cwuser.groups |
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
476 |
cwuser.properties |
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
477 |
return cwuser |
1482 | 478 |
|
0 | 479 |
# public (dbapi) interface ################################################ |
1482 | 480 |
|
5587
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
481 |
def stats(self): # XXX restrict to managers session? |
6256
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
482 |
"""Return a dictionary containing some statistics about the repository |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
483 |
resources usage. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
484 |
|
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
485 |
This is a public method, not requiring a session id. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
486 |
""" |
5587
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
487 |
results = {} |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
488 |
querier = self.querier |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
489 |
source = self.system_source |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
490 |
for size, maxsize, hits, misses, title in ( |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
491 |
(len(querier._rql_cache), self.config['rql-cache-size'], |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
492 |
querier.cache_hit, querier.cache_miss, 'rqlt_st'), |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
493 |
(len(source._cache), self.config['rql-cache-size'], |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
494 |
source.cache_hit, source.cache_miss, 'sql'), |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
495 |
): |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
496 |
results['%s_cache_size' % title] = '%s / %s' % (size, maxsize) |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
497 |
results['%s_cache_hit' % title] = hits |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
498 |
results['%s_cache_miss' % title] = misses |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
499 |
results['%s_cache_hit_percent' % title] = (hits * 100) / (hits + misses) |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
500 |
results['sql_no_cache'] = self.system_source.no_cache |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
501 |
results['nb_open_sessions'] = len(self._sessions) |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
502 |
results['nb_active_threads'] = threading.activeCount() |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
503 |
results['looping_tasks'] = ', '.join(str(t) for t in self._looping_tasks) |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
504 |
results['available_cnxsets'] = self._cnxsets_pool.qsize() |
5606
61b28589d33f
merge back to stable some changes made on site for a customer.
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
diff
changeset
|
505 |
results['threads'] = ', '.join(sorted(str(t) for t in threading.enumerate())) |
5587
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
506 |
return results |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
507 |
|
0 | 508 |
def get_schema(self): |
6256
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
509 |
"""Return the instance schema. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
510 |
|
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
511 |
This is a public method, not requiring a session id. |
0 | 512 |
""" |
513 |
try: |
|
514 |
# necessary to support pickling used by pyro |
|
515 |
self.schema.__hashmode__ = 'pickle' |
|
516 |
return self.schema |
|
517 |
finally: |
|
518 |
self.schema.__hashmode__ = None |
|
519 |
||
520 |
def get_cubes(self): |
|
6256
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
521 |
"""Return the list of cubes used by this instance. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
522 |
|
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
523 |
This is a public method, not requiring a session id. |
0 | 524 |
""" |
2473
490f88fb99b6
new distinguish repairing/creating from regular start.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2268
diff
changeset
|
525 |
versions = self.get_versions(not (self.config.creating |
4689
4eb1f4490538
[test] skipping versions checking during test is enough, no need for monkey patch
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4687
diff
changeset
|
526 |
or self.config.repairing |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
527 |
or self.config.quick_start |
4689
4eb1f4490538
[test] skipping versions checking during test is enough, no need for monkey patch
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4687
diff
changeset
|
528 |
or self.config.mode == 'test')) |
0 | 529 |
cubes = list(versions) |
530 |
cubes.remove('cubicweb') |
|
531 |
return cubes |
|
532 |
||
6308
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
533 |
def get_option_value(self, option, foreid=None): |
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
534 |
"""Return the value for `option` in the configuration. If `foreid` is |
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
535 |
specified, the actual repository to which this entity belongs is |
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
536 |
derefenced and the option value retrieved from it. |
6257
7eb5f1aed728
[repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6256
diff
changeset
|
537 |
|
7eb5f1aed728
[repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6256
diff
changeset
|
538 |
This is a public method, not requiring a session id. |
7eb5f1aed728
[repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6256
diff
changeset
|
539 |
""" |
7eb5f1aed728
[repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6256
diff
changeset
|
540 |
# XXX we may want to check we don't give sensible information |
6308
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
541 |
if foreid is None: |
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
542 |
return self.config[option] |
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
543 |
_, sourceuri, extid = self.type_and_source_from_eid(foreid) |
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
544 |
if sourceuri == 'system': |
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
545 |
return self.config[option] |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
546 |
cnxset = self._get_cnxset() |
6308
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
547 |
try: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
548 |
cnx = cnxset.connection(sourceuri) |
7064
b8ce2266d026
[repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6889
diff
changeset
|
549 |
# needed to check connection is valid and usable by the current |
b8ce2266d026
[repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6889
diff
changeset
|
550 |
# thread |
b8ce2266d026
[repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6889
diff
changeset
|
551 |
newcnx = self.sources_by_uri[sourceuri].check_connection(cnx) |
b8ce2266d026
[repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6889
diff
changeset
|
552 |
if newcnx is not None: |
b8ce2266d026
[repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6889
diff
changeset
|
553 |
cnx = newcnx |
b8ce2266d026
[repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6889
diff
changeset
|
554 |
return cnx.get_option_value(option, extid) |
6308
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
555 |
finally: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
556 |
self._free_cnxset(cnxset) |
6257
7eb5f1aed728
[repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6256
diff
changeset
|
557 |
|
0 | 558 |
@cached |
559 |
def get_versions(self, checkversions=False): |
|
6256
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
560 |
"""Return the a dictionary containing cubes used by this instance |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
561 |
as key with their version as value, including cubicweb version. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
562 |
|
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
563 |
This is a public method, not requiring a session id. |
0 | 564 |
""" |
565 |
from logilab.common.changelog import Version |
|
566 |
vcconf = {} |
|
567 |
session = self.internal_session() |
|
568 |
try: |
|
569 |
for pk, version in session.execute( |
|
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
570 |
'Any K,V WHERE P is CWProperty, P value V, P pkey K, ' |
0 | 571 |
'P pkey ~="system.version.%"', build_descr=False): |
572 |
cube = pk.split('.')[-1] |
|
573 |
# XXX cubicweb migration |
|
574 |
if cube in CW_MIGRATION_MAP: |
|
575 |
cube = CW_MIGRATION_MAP[cube] |
|
576 |
version = Version(version) |
|
577 |
vcconf[cube] = version |
|
578 |
if checkversions: |
|
579 |
if cube != 'cubicweb': |
|
580 |
fsversion = self.config.cube_version(cube) |
|
581 |
else: |
|
582 |
fsversion = self.config.cubicweb_version() |
|
583 |
if version < fsversion: |
|
2476
1294a6bdf3bf
application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2473
diff
changeset
|
584 |
msg = ('instance has %s version %s but %s ' |
0 | 585 |
'is installed. Run "cubicweb-ctl upgrade".') |
586 |
raise ExecutionError(msg % (cube, version, fsversion)) |
|
587 |
finally: |
|
588 |
session.close() |
|
589 |
return vcconf |
|
1482 | 590 |
|
0 | 591 |
@cached |
592 |
def source_defs(self): |
|
6256
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
593 |
"""Return the a dictionary containing source uris as value and a |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
594 |
dictionary describing each source as value. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
595 |
|
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
596 |
This is a public method, not requiring a session id. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
597 |
""" |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
598 |
sources = {} |
0 | 599 |
# remove sensitive information |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
600 |
for uri, source in self.sources_by_uri.iteritems(): |
6722
3341521d857b
[repo source] rename attribute so it's much easier to grasp its role
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6674
diff
changeset
|
601 |
sources[uri] = source.public_config |
0 | 602 |
return sources |
603 |
||
604 |
def properties(self): |
|
6256
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
605 |
"""Return a result set containing system wide properties. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
606 |
|
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
607 |
This is a public method, not requiring a session id. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
608 |
""" |
0 | 609 |
session = self.internal_session() |
610 |
try: |
|
5747
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
611 |
# don't use session.execute, we don't want rset.req set |
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
612 |
return self.querier.execute(session, 'Any K,V WHERE P is CWProperty,' |
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
613 |
'P pkey K, P value V, NOT P for_user U', |
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
614 |
build_descr=False) |
0 | 615 |
finally: |
616 |
session.close() |
|
617 |
||
5032 | 618 |
# XXX protect this method: anonymous should be allowed and registration |
619 |
# plugged |
|
1372
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
620 |
def register_user(self, login, password, email=None, **kwargs): |
0 | 621 |
"""check a user with the given login exists, if not create it with the |
622 |
given password. This method is designed to be used for anonymous |
|
623 |
registration on public web site. |
|
624 |
""" |
|
625 |
session = self.internal_session() |
|
1372
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
626 |
# for consistency, keep same error as unique check hook (although not required) |
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
627 |
errmsg = session._('the value "%s" is already used, use another one') |
0 | 628 |
try: |
5073
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
629 |
if (session.execute('CWUser X WHERE X login %(login)s', {'login': login}, |
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
630 |
build_descr=False) |
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
631 |
or session.execute('CWUser X WHERE X use_email C, C address %(login)s', |
5073
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
632 |
{'login': login}, build_descr=False)): |
5030
5238d9a8dfee
[form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4984
diff
changeset
|
633 |
qname = role_name('login', 'subject') |
5238d9a8dfee
[form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4984
diff
changeset
|
634 |
raise ValidationError(None, {qname: errmsg % login}) |
0 | 635 |
# we have to create the user |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
636 |
user = self.vreg['etypes'].etype_class('CWUser')(session) |
0 | 637 |
if isinstance(password, unicode): |
638 |
# password should *always* be utf8 encoded |
|
639 |
password = password.encode('UTF8') |
|
640 |
kwargs['login'] = login |
|
641 |
kwargs['upassword'] = password |
|
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
642 |
self.glob_add_entity(session, EditedEntity(user, **kwargs)) |
0 | 643 |
session.execute('SET X in_group G WHERE X eid %(x)s, G name "users"', |
644 |
{'x': user.eid}) |
|
1372
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
645 |
if email or '@' in login: |
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
646 |
d = {'login': login, 'email': email or login} |
5073
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
647 |
if session.execute('EmailAddress X WHERE X address %(email)s', d, |
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
648 |
build_descr=False): |
5030
5238d9a8dfee
[form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4984
diff
changeset
|
649 |
qname = role_name('address', 'subject') |
5238d9a8dfee
[form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4984
diff
changeset
|
650 |
raise ValidationError(None, {qname: errmsg % d['email']}) |
1372
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
651 |
session.execute('INSERT EmailAddress X: X address %(email)s, ' |
5032 | 652 |
'U primary_email X, U use_email X ' |
5073
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
653 |
'WHERE U login %(login)s', d, build_descr=False) |
0 | 654 |
session.commit() |
655 |
finally: |
|
656 |
session.close() |
|
594
76218d42d21f
return success or not on creation of user
Arthur Lutz <arthur.lutz@logilab.fr>
parents:
479
diff
changeset
|
657 |
return True |
1482 | 658 |
|
6390
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
659 |
def find_users(self, fetch_attrs, **query_attrs): |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
660 |
"""yield user attributes for cwusers matching the given query_attrs |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
661 |
(the result set cannot survive this method call) |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
662 |
|
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
663 |
This can be used by low-privileges account (anonymous comes to |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
664 |
mind). |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
665 |
|
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
666 |
`fetch_attrs`: tuple of attributes to be fetched |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
667 |
`query_attrs`: dict of attr/values to restrict the query |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
668 |
""" |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
669 |
assert query_attrs |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
670 |
if not hasattr(self, '_cwuser_attrs'): |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
671 |
cwuser = self.schema['CWUser'] |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
672 |
self._cwuser_attrs = set(str(rschema) |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
673 |
for rschema, _eschema in cwuser.attribute_definitions() |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
674 |
if not rschema.meta) |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
675 |
cwuserattrs = self._cwuser_attrs |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
676 |
for k in chain(fetch_attrs, query_attrs.iterkeys()): |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
677 |
if k not in cwuserattrs: |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
678 |
raise Exception('bad input for find_user') |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
679 |
session = self.internal_session() |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
680 |
try: |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
681 |
varmaker = rqlvar_maker() |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
682 |
vars = [(attr, varmaker.next()) for attr in fetch_attrs] |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
683 |
rql = 'Any %s WHERE X is CWUser, ' % ','.join(var[1] for var in vars) |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
684 |
rql += ','.join('X %s %s' % (var[0], var[1]) for var in vars) + ',' |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
685 |
rset = session.execute(rql + ','.join('X %s %%(%s)s' % (attr, attr) |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
686 |
for attr in query_attrs.iterkeys()), |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
687 |
query_attrs) |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
688 |
return rset.rows |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
689 |
finally: |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
690 |
session.close() |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
691 |
|
3647
2941f4a0aab9
refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3629
diff
changeset
|
692 |
def connect(self, login, **kwargs): |
0 | 693 |
"""open a connection for a given user |
694 |
||
695 |
base_url may be needed to send mails |
|
696 |
cnxtype indicate if this is a pyro connection or a in-memory connection |
|
1482 | 697 |
|
0 | 698 |
raise `AuthenticationError` if the authentication failed |
699 |
raise `ConnectionError` if we can't open a connection |
|
700 |
""" |
|
701 |
# use an internal connection |
|
702 |
session = self.internal_session() |
|
703 |
# try to get a user object |
|
4089
ff92c7d692bf
typos, api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3890
diff
changeset
|
704 |
cnxprops = kwargs.pop('cnxprops', None) |
0 | 705 |
try: |
3647
2941f4a0aab9
refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3629
diff
changeset
|
706 |
user = self.authenticate_user(session, login, **kwargs) |
0 | 707 |
finally: |
708 |
session.close() |
|
709 |
session = Session(user, self, cnxprops) |
|
3379
9192ba07890d
use .cw_rset instead of rset on appobject classes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3293
diff
changeset
|
710 |
user._cw = user.cw_rset.req = session |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
711 |
user.cw_clear_relation_cache() |
0 | 712 |
self._sessions[session.id] = session |
5638
cea831403e72
log the user name on session opening for easier debugging
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5603
diff
changeset
|
713 |
self.info('opened session %s for user %s', session.id, login) |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
714 |
self.hm.call_hooks('session_open', session) |
0 | 715 |
# commit session at this point in case write operation has been done |
716 |
# during `session_open` hooks |
|
717 |
session.commit() |
|
718 |
return session.id |
|
719 |
||
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
720 |
def execute(self, sessionid, rqlstring, args=None, build_descr=True, |
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
721 |
txid=None): |
0 | 722 |
"""execute a RQL query |
723 |
||
724 |
* rqlstring should be an unicode string or a plain ascii string |
|
725 |
* args the optional parameters used in the query |
|
726 |
* build_descr is a flag indicating if the description should be |
|
727 |
built on select queries |
|
728 |
""" |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
729 |
session = self._get_session(sessionid, setcnxset=True, txid=txid) |
0 | 730 |
try: |
731 |
try: |
|
5747
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
732 |
rset = self.querier.execute(session, rqlstring, args, |
0 | 733 |
build_descr) |
5747
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
734 |
# NOTE: the web front will (re)build it when needed |
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
735 |
# e.g in facets |
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
736 |
# Zeroed to avoid useless overhead with pyro |
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
737 |
rset._rqlst = None |
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
738 |
return rset |
0 | 739 |
except (Unauthorized, RQLSyntaxError): |
740 |
raise |
|
741 |
except ValidationError, ex: |
|
742 |
# need ValidationError normalization here so error may pass |
|
743 |
# through pyro |
|
744 |
if hasattr(ex.entity, 'eid'): |
|
745 |
ex.entity = ex.entity.eid # error raised by yams |
|
746 |
args = list(ex.args) |
|
747 |
args[0] = ex.entity |
|
748 |
ex.args = tuple(args) |
|
749 |
raise |
|
750 |
except: |
|
751 |
# FIXME: check error to catch internal errors |
|
5167
529861a73ec8
log RQL query in case of unexpected failure (priceless when porting to new database)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5115
diff
changeset
|
752 |
self.exception('unexpected error while executing %s with %s', rqlstring, args) |
0 | 753 |
raise |
754 |
finally: |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
755 |
session.free_cnxset() |
1482 | 756 |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
757 |
def describe(self, sessionid, eid, txid=None): |
0 | 758 |
"""return a tuple (type, source, extid) for the entity with id <eid>""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
759 |
session = self._get_session(sessionid, setcnxset=True, txid=txid) |
0 | 760 |
try: |
761 |
return self.type_and_source_from_eid(eid, session) |
|
762 |
finally: |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
763 |
session.free_cnxset() |
0 | 764 |
|
765 |
def check_session(self, sessionid): |
|
6012
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
766 |
"""raise `BadConnectionId` if the connection is no more valid, else |
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
767 |
return its latest activity timestamp. |
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
768 |
""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
769 |
return self._get_session(sessionid, setcnxset=False).timestamp |
0 | 770 |
|
6013
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
771 |
def get_shared_data(self, sessionid, key, default=None, pop=False, txdata=False): |
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
772 |
"""return value associated to key in the session's data dictionary or |
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
773 |
session's transaction's data if `txdata` is true. |
0 | 774 |
|
6013
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
775 |
If pop is True, value will be removed from the dictionnary. |
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
776 |
|
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
777 |
If key isn't defined in the dictionnary, value specified by the |
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
778 |
`default` argument will be returned. |
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
779 |
""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
780 |
session = self._get_session(sessionid, setcnxset=False) |
6013
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
781 |
return session.get_shared_data(key, default, pop, txdata) |
0 | 782 |
|
6013
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
783 |
def set_shared_data(self, sessionid, key, value, txdata=False): |
0 | 784 |
"""set value associated to `key` in shared data |
785 |
||
6013
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
786 |
if `txdata` is true, the value will be added to the repository session's |
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
787 |
transaction's data which are cleared on commit/rollback of the current |
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
788 |
transaction. |
0 | 789 |
""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
790 |
session = self._get_session(sessionid, setcnxset=False) |
6013
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
791 |
session.set_shared_data(key, value, txdata) |
0 | 792 |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
793 |
def commit(self, sessionid, txid=None): |
0 | 794 |
"""commit transaction for the session with the given id""" |
795 |
self.debug('begin commit for session %s', sessionid) |
|
796 |
try: |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
797 |
session = self._get_session(sessionid) |
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
798 |
session.set_tx_data(txid) |
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
799 |
return session.commit() |
1482 | 800 |
except (ValidationError, Unauthorized): |
0 | 801 |
raise |
802 |
except: |
|
803 |
self.exception('unexpected error') |
|
804 |
raise |
|
1482 | 805 |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
806 |
def rollback(self, sessionid, txid=None): |
0 | 807 |
"""commit transaction for the session with the given id""" |
808 |
self.debug('begin rollback for session %s', sessionid) |
|
809 |
try: |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
810 |
session = self._get_session(sessionid) |
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
811 |
session.set_tx_data(txid) |
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
812 |
session.rollback() |
0 | 813 |
except: |
814 |
self.exception('unexpected error') |
|
815 |
raise |
|
816 |
||
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
817 |
def close(self, sessionid, txid=None, checkshuttingdown=True): |
0 | 818 |
"""close the session with the given id""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
819 |
session = self._get_session(sessionid, setcnxset=True, txid=txid, |
1939
67e7379edd96
#343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
1883
diff
changeset
|
820 |
checkshuttingdown=checkshuttingdown) |
0 | 821 |
# operation uncommited before close are rollbacked before hook is called |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
822 |
session.rollback(free_cnxset=False) |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
823 |
self.hm.call_hooks('session_close', session) |
0 | 824 |
# commit session at this point in case write operation has been done |
825 |
# during `session_close` hooks |
|
826 |
session.commit() |
|
827 |
session.close() |
|
828 |
del self._sessions[sessionid] |
|
829 |
self.info('closed session %s for user %s', sessionid, session.user.login) |
|
1482 | 830 |
|
0 | 831 |
def user_info(self, sessionid, props=None): |
832 |
"""this method should be used by client to: |
|
833 |
* check session id validity |
|
834 |
* update user information on each user's request (i.e. groups and |
|
835 |
custom properties) |
|
836 |
""" |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
837 |
session = self._get_session(sessionid, setcnxset=False) |
2245
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
838 |
if props is not None: |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
839 |
self.set_session_props(sessionid, props) |
0 | 840 |
user = session.user |
841 |
return user.eid, user.login, user.groups, user.properties |
|
1482 | 842 |
|
2245
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
843 |
def set_session_props(self, sessionid, props): |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
844 |
"""this method should be used by client to: |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
845 |
* check session id validity |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
846 |
* update user information on each user's request (i.e. groups and |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
847 |
custom properties) |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
848 |
""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
849 |
session = self._get_session(sessionid, setcnxset=False) |
2245
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
850 |
for prop, value in props.items(): |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
851 |
session.change_property(prop, value) |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
852 |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
853 |
def undoable_transactions(self, sessionid, ueid=None, txid=None, |
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
854 |
**actionfilters): |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
855 |
"""See :class:`cubicweb.dbapi.Connection.undoable_transactions`""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
856 |
session = self._get_session(sessionid, setcnxset=True, txid=txid) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
857 |
try: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
858 |
return self.system_source.undoable_transactions(session, ueid, |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
859 |
**actionfilters) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
860 |
finally: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
861 |
session.free_cnxset() |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
862 |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
863 |
def transaction_info(self, sessionid, txuuid, txid=None): |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
864 |
"""See :class:`cubicweb.dbapi.Connection.transaction_info`""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
865 |
session = self._get_session(sessionid, setcnxset=True, txid=txid) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
866 |
try: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
867 |
return self.system_source.tx_info(session, txuuid) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
868 |
finally: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
869 |
session.free_cnxset() |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
870 |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
871 |
def transaction_actions(self, sessionid, txuuid, public=True, txid=None): |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
872 |
"""See :class:`cubicweb.dbapi.Connection.transaction_actions`""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
873 |
session = self._get_session(sessionid, setcnxset=True, txid=txid) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
874 |
try: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
875 |
return self.system_source.tx_actions(session, txuuid, public) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
876 |
finally: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
877 |
session.free_cnxset() |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
878 |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
879 |
def undo_transaction(self, sessionid, txuuid, txid=None): |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
880 |
"""See :class:`cubicweb.dbapi.Connection.undo_transaction`""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
881 |
session = self._get_session(sessionid, setcnxset=True, txid=txid) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
882 |
try: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
883 |
return self.system_source.undo_transaction(session, txuuid) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
884 |
finally: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
885 |
session.free_cnxset() |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
886 |
|
0 | 887 |
# public (inter-repository) interface ##################################### |
1482 | 888 |
|
0 | 889 |
def entities_modified_since(self, etypes, mtime): |
890 |
"""function designed to be called from an external repository which |
|
891 |
is using this one as a rql source for synchronization, and return a |
|
892 |
3-uple containing : |
|
893 |
* the local date |
|
894 |
* list of (etype, eid) of entities of the given types which have been |
|
895 |
modified since the given timestamp (actually entities whose full text |
|
896 |
index content has changed) |
|
897 |
* list of (etype, eid) of entities of the given types which have been |
|
898 |
deleted since the given timestamp |
|
899 |
""" |
|
900 |
session = self.internal_session() |
|
1016
26387b836099
use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents:
636
diff
changeset
|
901 |
updatetime = datetime.now() |
0 | 902 |
try: |
903 |
modentities, delentities = self.system_source.modified_entities( |
|
904 |
session, etypes, mtime) |
|
905 |
return updatetime, modentities, delentities |
|
906 |
finally: |
|
907 |
session.close() |
|
908 |
||
909 |
# session handling ######################################################## |
|
1482 | 910 |
|
0 | 911 |
def close_sessions(self): |
912 |
"""close every opened sessions""" |
|
913 |
for sessionid in self._sessions.keys(): |
|
914 |
try: |
|
1939
67e7379edd96
#343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
1883
diff
changeset
|
915 |
self.close(sessionid, checkshuttingdown=False) |
0 | 916 |
except: |
917 |
self.exception('error while closing session %s' % sessionid) |
|
918 |
||
919 |
def clean_sessions(self): |
|
920 |
"""close sessions not used since an amount of time specified in the |
|
921 |
configuration |
|
922 |
""" |
|
6012
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
923 |
mintime = time() - self.cleanup_session_time |
0 | 924 |
self.debug('cleaning session unused since %s', |
925 |
strftime('%T', localtime(mintime))) |
|
926 |
nbclosed = 0 |
|
927 |
for session in self._sessions.values(): |
|
928 |
if session.timestamp < mintime: |
|
929 |
self.close(session.id) |
|
930 |
nbclosed += 1 |
|
931 |
return nbclosed |
|
1482 | 932 |
|
0 | 933 |
def internal_session(self, cnxprops=None): |
934 |
"""return a dbapi like connection/cursor using internal user which |
|
935 |
have every rights on the repository. You'll *have to* commit/rollback |
|
936 |
or close (rollback implicitly) the session once the job's done, else |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
937 |
you'll leak connections set up to the time where no one is |
0 | 938 |
available, causing irremediable freeze... |
939 |
""" |
|
940 |
session = InternalSession(self, cnxprops) |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
941 |
session.set_cnxset() |
0 | 942 |
return session |
1482 | 943 |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
944 |
def _get_session(self, sessionid, setcnxset=False, txid=None, |
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
945 |
checkshuttingdown=True): |
0 | 946 |
"""return the user associated to the given session identifier""" |
5906
d40ced753291
[repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5813
diff
changeset
|
947 |
if checkshuttingdown and self.shutting_down: |
1883
011e13d74cfc
shuting -> shutting
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
1880
diff
changeset
|
948 |
raise Exception('Repository is shutting down') |
0 | 949 |
try: |
950 |
session = self._sessions[sessionid] |
|
951 |
except KeyError: |
|
952 |
raise BadConnectionId('No such session %s' % sessionid) |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
953 |
if setcnxset: |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
954 |
session.set_tx_data(txid) # must be done before set_cnxset |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
955 |
session.set_cnxset() |
0 | 956 |
return session |
957 |
||
958 |
# data sources handling ################################################### |
|
959 |
# * correspondance between eid and (type, source) |
|
960 |
# * correspondance between eid and local id (i.e. specific to a given source) |
|
1482 | 961 |
|
0 | 962 |
def type_and_source_from_eid(self, eid, session=None): |
963 |
"""return a tuple (type, source, extid) for the entity with id <eid>""" |
|
964 |
try: |
|
965 |
eid = typed_eid(eid) |
|
966 |
except ValueError: |
|
967 |
raise UnknownEid(eid) |
|
968 |
try: |
|
969 |
return self._type_source_cache[eid] |
|
970 |
except KeyError: |
|
971 |
if session is None: |
|
972 |
session = self.internal_session() |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
973 |
free_cnxset = True |
0 | 974 |
else: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
975 |
free_cnxset = False |
0 | 976 |
try: |
977 |
etype, uri, extid = self.system_source.eid_type_source(session, |
|
978 |
eid) |
|
979 |
finally: |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
980 |
if free_cnxset: |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
981 |
session.free_cnxset() |
0 | 982 |
self._type_source_cache[eid] = (etype, uri, extid) |
983 |
if uri != 'system': |
|
984 |
self._extid_cache[(extid, uri)] = eid |
|
985 |
return etype, uri, extid |
|
986 |
||
987 |
def clear_caches(self, eids): |
|
988 |
etcache = self._type_source_cache |
|
989 |
extidcache = self._extid_cache |
|
990 |
rqlcache = self.querier._rql_cache |
|
991 |
for eid in eids: |
|
992 |
try: |
|
993 |
etype, uri, extid = etcache.pop(typed_eid(eid)) # may be a string in some cases |
|
994 |
rqlcache.pop('%s X WHERE X eid %s' % (etype, eid), None) |
|
995 |
extidcache.pop((extid, uri), None) |
|
996 |
except KeyError: |
|
997 |
etype = None |
|
998 |
rqlcache.pop('Any X WHERE X eid %s' % eid, None) |
|
999 |
for source in self.sources: |
|
1000 |
source.clear_eid_cache(eid, etype) |
|
1482 | 1001 |
|
0 | 1002 |
def type_from_eid(self, eid, session=None): |
1003 |
"""return the type of the entity with id <eid>""" |
|
1004 |
return self.type_and_source_from_eid(eid, session)[0] |
|
1482 | 1005 |
|
0 | 1006 |
def source_from_eid(self, eid, session=None): |
1007 |
"""return the source for the given entity's eid""" |
|
1008 |
return self.sources_by_uri[self.type_and_source_from_eid(eid, session)[1]] |
|
1482 | 1009 |
|
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1010 |
def querier_cache_key(self, session, rql, args, eidkeys): |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1011 |
cachekey = [rql] |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1012 |
for key in sorted(eidkeys): |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1013 |
try: |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1014 |
etype = self.type_from_eid(args[key], session) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1015 |
except KeyError: |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1016 |
raise QueryError('bad cache key %s (no value)' % key) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1017 |
except TypeError: |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1018 |
raise QueryError('bad cache key %s (value: %r)' % ( |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1019 |
key, args[key])) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1020 |
cachekey.append(etype) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1021 |
# ensure eid is correctly typed in args |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1022 |
args[key] = typed_eid(args[key]) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1023 |
return tuple(cachekey) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1024 |
|
0 | 1025 |
def eid2extid(self, source, eid, session=None): |
1026 |
"""get local id from an eid""" |
|
1027 |
etype, uri, extid = self.type_and_source_from_eid(eid, session) |
|
1028 |
if source.uri != uri: |
|
1029 |
# eid not from the given source |
|
1030 |
raise UnknownEid(eid) |
|
1031 |
return extid |
|
1032 |
||
1954 | 1033 |
def extid2eid(self, source, extid, etype, session=None, insert=True, |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1034 |
sourceparams=None): |
7399
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1035 |
"""Return eid from a local id. If the eid is a negative integer, that |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1036 |
means the entity is known but has been copied back to the system source |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1037 |
hence should be ignored. |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1038 |
|
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1039 |
If no record is found, ie the entity is not known yet: |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1040 |
|
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1041 |
1. an eid is attributed |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1042 |
|
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1043 |
2. the source's :meth:`before_entity_insertion` method is called to |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1044 |
build the entity instance |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1045 |
|
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1046 |
3. unless source's :attr:`should_call_hooks` tell otherwise, |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1047 |
'before_add_entity' hooks are called |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1048 |
|
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1049 |
4. record is added into the system source |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1050 |
|
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1051 |
5. the source's :meth:`after_entity_insertion` method is called to |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1052 |
complete building of the entity instance |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1053 |
|
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1054 |
6. unless source's :attr:`should_call_hooks` tell otherwise, |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1055 |
'before_add_entity' hooks are called |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1056 |
""" |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1057 |
uri = 'system' if source.copy_based_source else source.uri |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1058 |
cachekey = (extid, uri) |
0 | 1059 |
try: |
1060 |
return self._extid_cache[cachekey] |
|
1061 |
except KeyError: |
|
1062 |
pass |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1063 |
free_cnxset = False |
0 | 1064 |
if session is None: |
1065 |
session = self.internal_session() |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1066 |
free_cnxset = True |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1067 |
eid = self.system_source.extid2eid(session, uri, extid) |
0 | 1068 |
if eid is not None: |
1069 |
self._extid_cache[cachekey] = eid |
|
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1070 |
self._type_source_cache[eid] = (etype, uri, extid) |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1071 |
if free_cnxset: |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1072 |
session.free_cnxset() |
0 | 1073 |
return eid |
1074 |
if not insert: |
|
1075 |
return |
|
1954 | 1076 |
# no link between extid and eid, create one using an internal session |
0 | 1077 |
# since the current session user may not have required permissions to |
1078 |
# do necessary stuff and we don't want to commit user session. |
|
1079 |
# |
|
450
5e14ea0e81c8
a note for later
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
341
diff
changeset
|
1080 |
# Moreover, even if session is already an internal session but is |
0 | 1081 |
# processing a commit, we have to use another one |
1082 |
if not session.is_internal_session: |
|
1083 |
session = self.internal_session() |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1084 |
free_cnxset = True |
0 | 1085 |
try: |
1086 |
eid = self.system_source.create_eid(session) |
|
1087 |
self._extid_cache[cachekey] = eid |
|
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1088 |
self._type_source_cache[eid] = (etype, uri, extid) |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1089 |
entity = source.before_entity_insertion( |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1090 |
session, extid, etype, eid, sourceparams) |
0 | 1091 |
if source.should_call_hooks: |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1092 |
self.hm.call_hooks('before_add_entity', session, entity=entity) |
450
5e14ea0e81c8
a note for later
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
341
diff
changeset
|
1093 |
# XXX call add_info with complete=False ? |
1954 | 1094 |
self.add_info(session, entity, source, extid) |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1095 |
source.after_entity_insertion(session, extid, entity, sourceparams) |
0 | 1096 |
if source.should_call_hooks: |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1097 |
self.hm.call_hooks('after_add_entity', session, entity=entity) |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1098 |
session.commit(free_cnxset) |
0 | 1099 |
return eid |
1100 |
except: |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1101 |
session.rollback(free_cnxset) |
0 | 1102 |
raise |
1482 | 1103 |
|
0 | 1104 |
def add_info(self, session, entity, source, extid=None, complete=True): |
1105 |
"""add type and source info for an eid into the system table, |
|
1106 |
and index the entity with the full text index |
|
1107 |
""" |
|
1108 |
# begin by inserting eid/type/source/extid into the entities table |
|
6426
541659c39f6a
[hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6390
diff
changeset
|
1109 |
hook.CleanupNewEidsCacheOp.get_instance(session).add_data(entity.eid) |
4806
4f12f59b1a13
[fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4756
diff
changeset
|
1110 |
self.system_source.add_info(session, entity, source, extid, complete) |
1482 | 1111 |
|
6953
260f662be125
[repo, ms] fix pb introduced in 6941:9ed02daa7dbb resulting in missing relation deletes when an entity is deleted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6945
diff
changeset
|
1112 |
def delete_info(self, session, entity, sourceuri, extid, scleanup=None): |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1113 |
"""called by external source when some entity known by the system source |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1114 |
has been deleted in the external source |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1115 |
""" |
5067
adc2122eed03
[repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5043
diff
changeset
|
1116 |
# mark eid as being deleted in session info and setup cache update |
adc2122eed03
[repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5043
diff
changeset
|
1117 |
# operation |
6426
541659c39f6a
[hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6390
diff
changeset
|
1118 |
hook.CleanupDeletedEidsCacheOp.get_instance(session).add_data(entity.eid) |
6626
5c20a005bddc
[pyro source] when cleaning relations of an external entity being cleaned up, only delete local relationsd
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6491
diff
changeset
|
1119 |
self._delete_info(session, entity, sourceuri, extid, scleanup) |
1482 | 1120 |
|
6953
260f662be125
[repo, ms] fix pb introduced in 6941:9ed02daa7dbb resulting in missing relation deletes when an entity is deleted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6945
diff
changeset
|
1121 |
def delete_info_multi(self, session, entities, sourceuri, extids, scleanup=None): |
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1122 |
"""same as delete_info but accepts a list of entities and |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1123 |
extids with the same etype and belonging to the same source |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1124 |
""" |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1125 |
# mark eid as being deleted in session info and setup cache update |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1126 |
# operation |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1127 |
op = hook.CleanupDeletedEidsCacheOp.get_instance(session) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1128 |
for entity in entities: |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1129 |
op.add_data(entity.eid) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1130 |
self._delete_info_multi(session, entities, sourceuri, extids, scleanup) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1131 |
|
6941
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1132 |
def _delete_info(self, session, entity, sourceuri, extid, scleanup=None): |
0 | 1133 |
"""delete system information on deletion of an entity: |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1134 |
* delete all remaining relations from/to this entity |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1135 |
* call delete info on the system source which will transfer record from |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1136 |
the entities table to the deleted_entities table |
0 | 1137 |
""" |
2641
9c33d98a074e
R [schema hooks] big refactoring / reorganization for clearer code, a few fixes on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2631
diff
changeset
|
1138 |
pendingrtypes = session.transaction_data.get('pendingrtypes', ()) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1139 |
# delete remaining relations: if user can delete the entity, he can |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1140 |
# delete all its relations without security checking |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
1141 |
with security_enabled(session, read=False, write=False): |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1142 |
eid = entity.eid |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1143 |
for rschema, _, role in entity.e_schema.relation_definitions(): |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
1144 |
rtype = rschema.type |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
1145 |
if rtype in schema.VIRTUAL_RTYPES or rtype in pendingrtypes: |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
1146 |
continue |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1147 |
if role == 'subject': |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
1148 |
# don't skip inlined relation so they are regularly |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
1149 |
# deleted and so hooks are correctly called |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1150 |
rql = 'DELETE X %s Y WHERE X eid %%(x)s' % rtype |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
1151 |
else: |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1152 |
rql = 'DELETE Y %s X WHERE X eid %%(x)s' % rtype |
6941
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1153 |
if scleanup is not None: |
6626
5c20a005bddc
[pyro source] when cleaning relations of an external entity being cleaned up, only delete local relationsd
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6491
diff
changeset
|
1154 |
# source cleaning: only delete relations stored locally |
6941
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1155 |
# (here, scleanup |
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1156 |
rql += ', NOT (Y cw_source S, S eid %(seid)s)' |
6674
fd9c76196bf7
[repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6664
diff
changeset
|
1157 |
try: |
6941
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1158 |
session.execute(rql, {'x': eid, 'seid': scleanup}, |
6674
fd9c76196bf7
[repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6664
diff
changeset
|
1159 |
build_descr=False) |
fd9c76196bf7
[repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6664
diff
changeset
|
1160 |
except: |
fd9c76196bf7
[repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6664
diff
changeset
|
1161 |
self.exception('error while cascading delete for entity %s ' |
fd9c76196bf7
[repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6664
diff
changeset
|
1162 |
'from %s. RQL: %s', entity, sourceuri, rql) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1163 |
self.system_source.delete_info(session, entity, sourceuri, extid) |
0 | 1164 |
|
6953
260f662be125
[repo, ms] fix pb introduced in 6941:9ed02daa7dbb resulting in missing relation deletes when an entity is deleted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6945
diff
changeset
|
1165 |
def _delete_info_multi(self, session, entities, sourceuri, extids, scleanup=None): |
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1166 |
"""same as _delete_info but accepts a list of entities with |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1167 |
the same etype and belinging to the same source. |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1168 |
""" |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1169 |
pendingrtypes = session.transaction_data.get('pendingrtypes', ()) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1170 |
# delete remaining relations: if user can delete the entity, he can |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1171 |
# delete all its relations without security checking |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1172 |
assert entities and len(entities) == len(extids) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1173 |
with security_enabled(session, read=False, write=False): |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1174 |
eids = [_e.eid for _e in entities] |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1175 |
in_eids = ','.join((str(eid) for eid in eids)) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1176 |
for rschema, _, role in entities[0].e_schema.relation_definitions(): |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1177 |
rtype = rschema.type |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1178 |
if rtype in schema.VIRTUAL_RTYPES or rtype in pendingrtypes: |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1179 |
continue |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1180 |
if role == 'subject': |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1181 |
# don't skip inlined relation so they are regularly |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1182 |
# deleted and so hooks are correctly called |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1183 |
rql = 'DELETE X %s Y WHERE X eid IN (%s)' % (rtype, in_eids) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1184 |
else: |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1185 |
rql = 'DELETE Y %s X WHERE X eid IN (%s)' % (rtype, in_eids) |
6941
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1186 |
if scleanup is not None: |
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1187 |
# source cleaning: only delete relations stored locally |
6941
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1188 |
rql += ', NOT (Y cw_source S, S eid %(seid)s)' |
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1189 |
try: |
6941
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1190 |
session.execute(rql, {'seid': scleanup}, build_descr=False) |
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1191 |
except: |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1192 |
self.exception('error while cascading delete for entity %s ' |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1193 |
'from %s. RQL: %s', entities, sourceuri, rql) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1194 |
self.system_source.delete_info_multi(session, entities, sourceuri, extids) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1195 |
|
0 | 1196 |
def locate_relation_source(self, session, subject, rtype, object): |
1197 |
subjsource = self.source_from_eid(subject, session) |
|
1198 |
objsource = self.source_from_eid(object, session) |
|
3042
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1199 |
if not subjsource is objsource: |
0 | 1200 |
source = self.system_source |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1201 |
if not (subjsource.may_cross_relation(rtype) |
3042
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1202 |
and objsource.may_cross_relation(rtype)): |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1203 |
raise MultiSourcesError( |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1204 |
"relation %s can't be crossed among sources" |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1205 |
% rtype) |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1206 |
elif not subjsource.support_relation(rtype): |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1207 |
source = self.system_source |
0 | 1208 |
else: |
1209 |
source = subjsource |
|
3042
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1210 |
if not source.support_relation(rtype, True): |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1211 |
raise MultiSourcesError( |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1212 |
"source %s doesn't support write of %s relation" |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1213 |
% (source.uri, rtype)) |
0 | 1214 |
return source |
1482 | 1215 |
|
0 | 1216 |
def locate_etype_source(self, etype): |
1217 |
for source in self.sources: |
|
1218 |
if source.support_entity(etype, 1): |
|
1219 |
return source |
|
1220 |
else: |
|
1221 |
raise ETypeNotSupportedBySources(etype) |
|
1482 | 1222 |
|
5068
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1223 |
def init_entity_caches(self, session, entity, source): |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1224 |
"""add entity to session entities cache and repo's extid cache. |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1225 |
Return entity's ext id if the source isn't the system source. |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1226 |
""" |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1227 |
session.set_entity_cache(entity) |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1228 |
suri = source.uri |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1229 |
if suri == 'system': |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1230 |
extid = None |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1231 |
else: |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1232 |
if source.copy_based_source: |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1233 |
suri = 'system' |
5068
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1234 |
extid = source.get_extid(entity) |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1235 |
self._extid_cache[(str(extid), suri)] = entity.eid |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1236 |
self._type_source_cache[entity.eid] = (entity.__regid__, suri, extid) |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1237 |
return extid |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1238 |
|
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1239 |
def glob_add_entity(self, session, edited): |
0 | 1240 |
"""add an entity to the repository |
1482 | 1241 |
|
0 | 1242 |
the entity eid should originaly be None and a unique eid is assigned to |
1243 |
the entity instance |
|
1244 |
""" |
|
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1245 |
entity = edited.entity |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1246 |
entity._cw_is_saved = False # entity has an eid but is not yet saved |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1247 |
# init edited_attributes before calling before_add_entity hooks |
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1248 |
entity.cw_edited = edited |
0 | 1249 |
eschema = entity.e_schema |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1250 |
source = self.locate_etype_source(entity.__regid__) |
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1251 |
# allocate an eid to the entity before calling hooks |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1252 |
entity.eid = self.system_source.create_eid(session) |
5068
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1253 |
# set caches asap |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1254 |
extid = self.init_entity_caches(session, entity, source) |
2600
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1255 |
if server.DEBUG & server.DBG_REPO: |
6466
43d71dd8a8ec
cleanup and 3.10 updates
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6465
diff
changeset
|
1256 |
print 'ADD entity', self, entity.__regid__, entity.eid, edited |
0 | 1257 |
relations = [] |
2929
51cdfe069e10
fix edited_attributes handling when adding entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2875
diff
changeset
|
1258 |
if source.should_call_hooks: |
2968
0e3460341023
somewhat painful backport of 3.5 branch, should mostly be ok
Sylvain Thénault <sylvain.thenault@logilab.fr>
diff
changeset
|
1259 |
self.hm.call_hooks('before_add_entity', session, entity=entity) |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1260 |
for attr in edited.iterkeys(): |
3689
deb13e88e037
follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3606
diff
changeset
|
1261 |
rschema = eschema.subjrels[attr] |
deb13e88e037
follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3606
diff
changeset
|
1262 |
if not rschema.final: # inlined relation |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1263 |
relations.append((attr, edited[attr])) |
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1264 |
edited.set_defaults() |
4843
5f7363416765
fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4835
diff
changeset
|
1265 |
if session.is_hook_category_activated('integrity'): |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1266 |
edited.check(creation=True) |
6464
11f9fbf6a645
[repo, entity] move entity cache initialization to a function, and call it before source.add_entity so it may be used in error handler or such
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6427
diff
changeset
|
1267 |
prefill_entity_caches(entity, relations) |
6211
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1268 |
try: |
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1269 |
source.add_entity(session, entity) |
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1270 |
except UniqueTogetherError, exc: |
6465
6401a9d0b5aa
[architecture] introduce adapter to ease transformation of errors before display to the end user. Use it for UniqueTogetherError first
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6464
diff
changeset
|
1271 |
userhdlr = session.vreg['adapters'].select( |
6401a9d0b5aa
[architecture] introduce adapter to ease transformation of errors before display to the end user. Use it for UniqueTogetherError first
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6464
diff
changeset
|
1272 |
'IUserFriendlyError', session, entity=entity, exc=exc) |
6401a9d0b5aa
[architecture] introduce adapter to ease transformation of errors before display to the end user. Use it for UniqueTogetherError first
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6464
diff
changeset
|
1273 |
userhdlr.raise_user_exception() |
0 | 1274 |
self.add_info(session, entity, source, extid, complete=False) |
6368
f907cc7f2875
[repo] properly mark object as saved once added to its source (code much probably gone during a merge...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6333
diff
changeset
|
1275 |
edited.saved = entity._cw_is_saved = True |
0 | 1276 |
# trigger after_add_entity after after_add_relation |
1277 |
if source.should_call_hooks: |
|
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1278 |
self.hm.call_hooks('after_add_entity', session, entity=entity) |
0 | 1279 |
# call hooks for inlined relations |
1280 |
for attr, value in relations: |
|
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1281 |
self.hm.call_hooks('before_add_relation', session, |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1282 |
eidfrom=entity.eid, rtype=attr, eidto=value) |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1283 |
self.hm.call_hooks('after_add_relation', session, |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1284 |
eidfrom=entity.eid, rtype=attr, eidto=value) |
0 | 1285 |
return entity.eid |
1482 | 1286 |
|
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1287 |
def glob_update_entity(self, session, edited): |
0 | 1288 |
"""replace an entity in the repository |
1289 |
the type and the eid of an entity must not be changed |
|
1290 |
""" |
|
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1291 |
entity = edited.entity |
2600
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1292 |
if server.DEBUG & server.DBG_REPO: |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1293 |
print 'UPDATE entity', entity.__regid__, entity.eid, \ |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1294 |
entity.cw_attr_cache, edited |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1295 |
hm = self.hm |
0 | 1296 |
eschema = entity.e_schema |
2647
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1297 |
session.set_entity_cache(entity) |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1298 |
orig_edited = getattr(entity, 'cw_edited', None) |
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1299 |
entity.cw_edited = edited |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1300 |
try: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1301 |
only_inline_rels, need_fti_update = True, False |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1302 |
relations = [] |
5292
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1303 |
source = self.source_from_eid(entity.eid, session) |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1304 |
for attr in list(edited): |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1305 |
if attr == 'eid': |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1306 |
continue |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1307 |
rschema = eschema.subjrels[attr] |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1308 |
if rschema.final: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1309 |
if getattr(eschema.rdef(attr), 'fulltextindexed', False): |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1310 |
need_fti_update = True |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1311 |
only_inline_rels = False |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1312 |
else: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1313 |
# inlined relation |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1314 |
previous_value = entity.related(attr) or None |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1315 |
if previous_value is not None: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1316 |
previous_value = previous_value[0][0] # got a result set |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1317 |
if previous_value == entity.cw_attr_cache[attr]: |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1318 |
previous_value = None |
5292
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1319 |
elif source.should_call_hooks: |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1320 |
hm.call_hooks('before_delete_relation', session, |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1321 |
eidfrom=entity.eid, rtype=attr, |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1322 |
eidto=previous_value) |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1323 |
relations.append((attr, edited[attr], previous_value)) |
5292
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1324 |
if source.should_call_hooks: |
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1325 |
# call hooks for inlined relations |
6211
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1326 |
for attr, value, _t in relations: |
5292
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1327 |
hm.call_hooks('before_add_relation', session, |
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1328 |
eidfrom=entity.eid, rtype=attr, eidto=value) |
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1329 |
if not only_inline_rels: |
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1330 |
hm.call_hooks('before_update_entity', session, entity=entity) |
5808
2de32c0c293b
[repo] call entity.check() once before_update hooks has been called, to let them a chance to transform data
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5749
diff
changeset
|
1331 |
if session.is_hook_category_activated('integrity'): |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1332 |
edited.check() |
6211
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1333 |
try: |
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1334 |
source.update_entity(session, entity) |
6225 | 1335 |
edited.saved = True |
6211
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1336 |
except UniqueTogetherError, exc: |
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1337 |
etype, rtypes = exc.args |
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1338 |
problems = {} |
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1339 |
for col in rtypes: |
6225 | 1340 |
problems[col] = session._('violates unique_together constraints (%s)') % (','.join(rtypes)) |
6211
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1341 |
raise ValidationError(entity.eid, problems) |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1342 |
self.system_source.update_info(session, entity, need_fti_update) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1343 |
if source.should_call_hooks: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1344 |
if not only_inline_rels: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1345 |
hm.call_hooks('after_update_entity', session, entity=entity) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1346 |
for attr, value, prevvalue in relations: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1347 |
# if the relation is already cached, update existant cache |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1348 |
relcache = entity.cw_relation_cached(attr, 'subject') |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1349 |
if prevvalue is not None: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1350 |
hm.call_hooks('after_delete_relation', session, |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1351 |
eidfrom=entity.eid, rtype=attr, eidto=prevvalue) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1352 |
if relcache is not None: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1353 |
session.update_rel_cache_del(entity.eid, attr, prevvalue) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1354 |
del_existing_rel_if_needed(session, entity.eid, attr, value) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1355 |
if relcache is not None: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1356 |
session.update_rel_cache_add(entity.eid, attr, value) |
2647
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1357 |
else: |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1358 |
entity.cw_set_relation_cache(attr, 'subject', |
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1359 |
session.eid_rset(value)) |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1360 |
hm.call_hooks('after_add_relation', session, |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1361 |
eidfrom=entity.eid, rtype=attr, eidto=value) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1362 |
finally: |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1363 |
if orig_edited is not None: |
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1364 |
entity.cw_edited = orig_edited |
0 | 1365 |
|
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1366 |
|
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1367 |
def glob_delete_entities(self, session, eids): |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1368 |
"""delete a list of entities and all related entities from the repository""" |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1369 |
data_by_etype_source = {} # values are ([list of eids], |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1370 |
# [list of extid], |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1371 |
# [list of entities]) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1372 |
# |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1373 |
# WARNING: the way this dictionary is populated is heavily optimized |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1374 |
# and does not use setdefault on purpose. Unless a new release |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1375 |
# of the Python interpreter advertises large perf improvements |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1376 |
# in setdefault, this should not be changed without profiling. |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1377 |
|
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1378 |
for eid in eids: |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1379 |
etype, sourceuri, extid = self.type_and_source_from_eid(eid, session) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1380 |
entity = session.entity_from_eid(eid, etype) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1381 |
_key = (etype, sourceuri) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1382 |
if _key not in data_by_etype_source: |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1383 |
data_by_etype_source[_key] = ([eid], [extid], [entity]) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1384 |
else: |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1385 |
_data = data_by_etype_source[_key] |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1386 |
_data[0].append(eid) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1387 |
_data[1].append(extid) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1388 |
_data[2].append(entity) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1389 |
for (etype, sourceuri), (eids, extids, entities) in data_by_etype_source.iteritems(): |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1390 |
if server.DEBUG & server.DBG_REPO: |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1391 |
print 'DELETE entities', etype, eids |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1392 |
#print 'DELETE entities', etype, len(eids) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1393 |
source = self.sources_by_uri[sourceuri] |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1394 |
if source.should_call_hooks: |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1395 |
self.hm.call_hooks('before_delete_entity', session, entities=entities) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1396 |
self._delete_info_multi(session, entities, sourceuri, extids) # xxx |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1397 |
source.delete_entities(session, entities) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1398 |
if source.should_call_hooks: |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1399 |
self.hm.call_hooks('after_delete_entity', session, entities=entities) |
0 | 1400 |
# don't clear cache here this is done in a hook on commit |
1482 | 1401 |
|
0 | 1402 |
def glob_add_relation(self, session, subject, rtype, object): |
1403 |
"""add a relation to the repository""" |
|
7237
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1404 |
self.glob_add_relations(session, {rtype: [(subject, object)]}) |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1405 |
|
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1406 |
def glob_add_relations(self, session, relations): |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1407 |
"""add several relations to the repository |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1408 |
|
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1409 |
relations is a dictionary rtype: [(subj_eid, obj_eid), ...] |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1410 |
""" |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1411 |
sources = {} |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1412 |
for rtype, eids_subj_obj in relations.iteritems(): |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1413 |
if server.DEBUG & server.DBG_REPO: |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1414 |
for subject, object in relations: |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1415 |
print 'ADD relation', subject, rtype, object |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1416 |
for subject, object in eids_subj_obj: |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1417 |
source = self.locate_relation_source(session, subject, rtype, object) |
7238
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1418 |
if source not in sources: |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1419 |
relations_by_rtype = {} |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1420 |
sources[source] = relations_by_rtype |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1421 |
else: |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1422 |
relations_by_rtype = sources[source] |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1423 |
if rtype in relations_by_rtype: |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1424 |
relations_by_rtype[rtype].append((subject, object)) |
7237
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1425 |
else: |
7238
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1426 |
relations_by_rtype[rtype] = [(subject, object)] |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1427 |
for source, relations_by_rtype in sources.iteritems(): |
7237
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1428 |
if source.should_call_hooks: |
7238
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1429 |
for rtype, source_relations in relations_by_rtype.iteritems(): |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1430 |
for subject, object in source_relations: |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1431 |
del_existing_rel_if_needed(session, subject, rtype, object) |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1432 |
self.hm.call_hooks('before_add_relation', session, |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1433 |
rtype=rtype, eids_from_to=source_relations) |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1434 |
for rtype, source_relations in relations_by_rtype.iteritems(): |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1435 |
source.add_relations(session, rtype, source_relations) |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1436 |
rschema = self.schema.rschema(rtype) |
7237
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1437 |
for subject, object in source_relations: |
7238
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1438 |
session.update_rel_cache_add(subject, rtype, object, rschema.symmetric) |
7237
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1439 |
if source.should_call_hooks: |
7238
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1440 |
for rtype, source_relations in relations_by_rtype.iteritems(): |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1441 |
self.hm.call_hooks('after_add_relation', session, |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1442 |
rtype=rtype, eids_from_to=source_relations) |
0 | 1443 |
|
1444 |
def glob_delete_relation(self, session, subject, rtype, object): |
|
1445 |
"""delete a relation from the repository""" |
|
2600
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1446 |
if server.DEBUG & server.DBG_REPO: |
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1447 |
print 'DELETE relation', subject, rtype, object |
0 | 1448 |
source = self.locate_relation_source(session, subject, rtype, object) |
1449 |
if source.should_call_hooks: |
|
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1450 |
self.hm.call_hooks('before_delete_relation', session, |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1451 |
eidfrom=subject, rtype=rtype, eidto=object) |
0 | 1452 |
source.delete_relation(session, subject, rtype, object) |
2647
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1453 |
rschema = self.schema.rschema(rtype) |
4467
0e73d299730a
fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4252
diff
changeset
|
1454 |
session.update_rel_cache_del(subject, rtype, object, rschema.symmetric) |
0e73d299730a
fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4252
diff
changeset
|
1455 |
if rschema.symmetric: |
0e73d299730a
fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4252
diff
changeset
|
1456 |
# on symmetric relation, we can't now in which sense it's |
0 | 1457 |
# stored so try to delete both |
1458 |
source.delete_relation(session, object, rtype, subject) |
|
1459 |
if source.should_call_hooks: |
|
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1460 |
self.hm.call_hooks('after_delete_relation', session, |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1461 |
eidfrom=subject, rtype=rtype, eidto=object) |
0 | 1462 |
|
1463 |
||
1464 |
# pyro handling ########################################################### |
|
1482 | 1465 |
|
6801
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1466 |
@property |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1467 |
@cached |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1468 |
def pyro_appid(self): |
6061
7cb29eab1c9d
[pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
1469 |
from logilab.common import pyro_ext as pyro |
7cb29eab1c9d
[pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
1470 |
config = self.config |
7cb29eab1c9d
[pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
1471 |
appid = '%s.%s' % pyro.ns_group_and_id( |
7cb29eab1c9d
[pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
1472 |
config['pyro-instance-id'] or config.appid, |
7cb29eab1c9d
[pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
1473 |
config['pyro-ns-group']) |
7cb29eab1c9d
[pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
1474 |
# ensure config['pyro-instance-id'] is a full qualified pyro name |
7cb29eab1c9d
[pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
1475 |
config['pyro-instance-id'] = appid |
6801
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1476 |
return appid |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1477 |
|
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1478 |
def _use_pyrons(self): |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1479 |
"""return True if the pyro-ns-host is set to something else |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1480 |
than NO_PYRONS, meaning we want to go through a pyro |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1481 |
nameserver""" |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1482 |
return self.config['pyro-ns-host'] != 'NO_PYRONS' |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1483 |
|
6801
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1484 |
def pyro_register(self, host=''): |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1485 |
"""register the repository as a pyro object""" |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1486 |
from logilab.common import pyro_ext as pyro |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1487 |
daemon = pyro.register_object(self, self.pyro_appid, |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1488 |
daemonhost=self.config['pyro-host'], |
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1489 |
nshost=self.config['pyro-ns-host'], |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1490 |
use_pyrons=self._use_pyrons()) |
6801
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1491 |
self.info('repository registered as a pyro object %s', self.pyro_appid) |
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1492 |
self.pyro_uri = pyro.get_object_uri(self.pyro_appid) |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1493 |
self.info('pyro uri is: %s', self.pyro_uri) |
0 | 1494 |
self.pyro_registered = True |
6801
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1495 |
# register a looping task to regularly ensure we're still registered |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1496 |
# into the pyro name server |
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1497 |
if self._use_pyrons(): |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1498 |
self.looping_task(60*10, self._ensure_pyro_ns) |
0 | 1499 |
return daemon |
1482 | 1500 |
|
6801
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1501 |
def _ensure_pyro_ns(self): |
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1502 |
if not self._use_pyrons(): |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1503 |
return |
6801
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1504 |
from logilab.common import pyro_ext as pyro |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1505 |
pyro.ns_reregister(self.pyro_appid, nshost=self.config['pyro-ns-host']) |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1506 |
self.info('repository re-registered as a pyro object %s', |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1507 |
self.pyro_appid) |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1508 |
|
1228
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1509 |
# multi-sources planner helpers ########################################### |
1482 | 1510 |
|
1228
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1511 |
@cached |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1512 |
def rel_type_sources(self, rtype): |
6128 | 1513 |
return tuple([source for source in self.sources |
1514 |
if source.support_relation(rtype) |
|
1515 |
or rtype in source.dont_cross_relations]) |
|
1482 | 1516 |
|
1228
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1517 |
@cached |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1518 |
def can_cross_relation(self, rtype): |
6128 | 1519 |
return tuple([source for source in self.sources |
1520 |
if source.support_relation(rtype) |
|
1521 |
and rtype in source.cross_relations]) |
|
1482 | 1522 |
|
1228
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1523 |
@cached |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1524 |
def is_multi_sources_relation(self, rtype): |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1525 |
return any(source for source in self.sources |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1526 |
if not source is self.system_source |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1527 |
and source.support_relation(rtype)) |
1482 | 1528 |
|
7083
b8e35cde46e9
help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7064
diff
changeset
|
1529 |
# these are overridden by set_log_methods below |
b8e35cde46e9
help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7064
diff
changeset
|
1530 |
# only defining here to prevent pylint from complaining |
b8e35cde46e9
help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7064
diff
changeset
|
1531 |
info = warning = error = critical = exception = debug = lambda msg,*a,**kw: None |
0 | 1532 |
|
1533 |
def pyro_unregister(config): |
|
1534 |
"""unregister the repository from the pyro name server""" |
|
2665
0c6281487f90
[pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2650
diff
changeset
|
1535 |
from logilab.common.pyro_ext import ns_unregister |
3539
f3b14d052798
[pyro] merge pyro-id / pyro-instance-id options, put all pyro options in the same section of the configuration file
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3250
diff
changeset
|
1536 |
appid = config['pyro-instance-id'] or config.appid |
2665
0c6281487f90
[pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2650
diff
changeset
|
1537 |
ns_unregister(appid, config['pyro-ns-group'], config['pyro-ns-host']) |
0 | 1538 |
|
1539 |
||
1540 |
from logging import getLogger |
|
1541 |
from cubicweb import set_log_methods |
|
1542 |
set_log_methods(Repository, getLogger('cubicweb.repository')) |