author | Sylvain Thénault <sylvain.thenault@logilab.fr> |
Thu, 22 Sep 2011 19:13:58 +0200 | |
branch | stable |
changeset 7838 | ea1360938033 |
parent 7823 | 5f145462e041 |
child 7845 | 2172978be237 |
child 7884 | 35d2e2f4e10a |
permissions | -rw-r--r-- |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1 |
# copyright 2003-2011 LOGILAB S.A. (Paris, FRANCE), all rights reserved. |
5421
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
2 |
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
3 |
# |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
4 |
# This file is part of CubicWeb. |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
5 |
# |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
6 |
# CubicWeb is free software: you can redistribute it and/or modify it under the |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
7 |
# terms of the GNU Lesser General Public License as published by the Free |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
8 |
# Software Foundation, either version 2.1 of the License, or (at your option) |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
9 |
# any later version. |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
10 |
# |
5424
8ecbcbff9777
replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5421
diff
changeset
|
11 |
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT |
5421
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
12 |
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
13 |
# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
14 |
# details. |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
15 |
# |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
16 |
# You should have received a copy of the GNU Lesser General Public License along |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
17 |
# with CubicWeb. If not, see <http://www.gnu.org/licenses/>. |
0 | 18 |
"""Defines the central class for the CubicWeb RQL server: the repository. |
19 |
||
20 |
The repository is an abstraction allowing execution of rql queries against |
|
21 |
data sources. Most of the work is actually done in helper classes. The |
|
22 |
repository mainly: |
|
23 |
||
24 |
* brings these classes all together to provide a single access |
|
2476
1294a6bdf3bf
application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2473
diff
changeset
|
25 |
point to a cubicweb instance. |
0 | 26 |
* handles session management |
27 |
* provides method for pyro registration, to call if pyro is enabled |
|
5587
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
28 |
""" |
0 | 29 |
|
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
30 |
from __future__ import with_statement |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
31 |
|
0 | 32 |
__docformat__ = "restructuredtext en" |
33 |
||
34 |
import sys |
|
5587
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
35 |
import threading |
0 | 36 |
import Queue |
7507
4c043afb104a
fix failures introduced by recent refactoring
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7503
diff
changeset
|
37 |
from warnings import warn |
6390
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
38 |
from itertools import chain |
4719
aaed3f813ef8
kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4714
diff
changeset
|
39 |
from os.path import join |
1016
26387b836099
use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents:
636
diff
changeset
|
40 |
from datetime import datetime |
0 | 41 |
from time import time, localtime, strftime |
42 |
||
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
43 |
from logilab.common.decorators import cached, clear_cache |
3757 | 44 |
from logilab.common.compat import any |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
45 |
from logilab.common import flatten |
0 | 46 |
|
47 |
from yams import BadSchemaDefinition |
|
5030
5238d9a8dfee
[form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4984
diff
changeset
|
48 |
from yams.schema import role_name |
0 | 49 |
from rql import RQLSyntaxError |
6390
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
50 |
from rql.utils import rqlvar_maker |
0 | 51 |
|
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
52 |
from cubicweb import (CW_SOFTWARE_ROOT, CW_MIGRATION_MAP, QueryError, |
2709
6ee816eb9f25
[hooksmanager] hooks should be reseted after vreg reload
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2708
diff
changeset
|
53 |
UnknownEid, AuthenticationError, ExecutionError, |
3042
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
54 |
ETypeNotSupportedBySources, MultiSourcesError, |
0 | 55 |
BadConnectionId, Unauthorized, ValidationError, |
6211
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
56 |
RepositoryError, UniqueTogetherError, typed_eid, onevent) |
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
57 |
from cubicweb import cwvreg, schema, server |
7573
c8f8762c986d
[repo, looping task] raise a custom exception when repository is shuting down, avoid looping task to be restarted in such case. Closes #1021276
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7550
diff
changeset
|
58 |
from cubicweb.server import ShuttingDown, utils, hook, pool, querier, sources |
5079
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
59 |
from cubicweb.server.session import Session, InternalSession, InternalManager, \ |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
60 |
security_enabled |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
61 |
from cubicweb.server.ssplanner import EditedEntity |
0 | 62 |
|
7782
40a49f4350a5
backout 7780:a1d5365fefc1
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7780
diff
changeset
|
63 |
NO_CACHE_RELATIONS = set( [('require_permission', 'object'), |
40a49f4350a5
backout 7780:a1d5365fefc1
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7780
diff
changeset
|
64 |
('owned_by', 'object'), |
7373
e5e6ef56cfb5
[repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7305
diff
changeset
|
65 |
('created_by', 'object'), |
e5e6ef56cfb5
[repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7305
diff
changeset
|
66 |
('cw_source', 'object'), |
e5e6ef56cfb5
[repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7305
diff
changeset
|
67 |
]) |
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
68 |
|
6467
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
69 |
def prefill_entity_caches(entity, relations): |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
70 |
session = entity._cw |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
71 |
# prefill entity relation caches |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
72 |
for rschema in entity.e_schema.subject_relations(): |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
73 |
rtype = str(rschema) |
7373
e5e6ef56cfb5
[repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7305
diff
changeset
|
74 |
if rtype in schema.VIRTUAL_RTYPES or (rtype, 'subject') in NO_CACHE_RELATIONS: |
6467
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
75 |
continue |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
76 |
if rschema.final: |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
77 |
entity.cw_attr_cache.setdefault(rtype, None) |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
78 |
else: |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
79 |
entity.cw_set_relation_cache(rtype, 'subject', |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
80 |
session.empty_rset()) |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
81 |
for rschema in entity.e_schema.object_relations(): |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
82 |
rtype = str(rschema) |
7373
e5e6ef56cfb5
[repo cache] there are some relations we don't want to cache, they may cause memory leak
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7305
diff
changeset
|
83 |
if rtype in schema.VIRTUAL_RTYPES or (rtype, 'object') in NO_CACHE_RELATIONS: |
6467
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
84 |
continue |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
85 |
entity.cw_set_relation_cache(rtype, 'object', session.empty_rset()) |
869813c2dd40
[entity, repo] move prefill_entity_cache function back to repository due to usage of del_existing_rel_if_needed function which is also defined there
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6466
diff
changeset
|
86 |
|
0 | 87 |
def del_existing_rel_if_needed(session, eidfrom, rtype, eidto): |
88 |
"""delete existing relation when adding a new one if card is 1 or ? |
|
89 |
||
90 |
have to be done once the new relation has been inserted to avoid having |
|
91 |
an entity without a relation for some time |
|
92 |
||
93 |
this kind of behaviour has to be done in the repository so we don't have |
|
94 |
hooks order hazardness |
|
95 |
""" |
|
7503
bc30c2faaadc
[repository] refactor and optimize '?1' relation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7502
diff
changeset
|
96 |
# skip that if integrity explicitly disabled |
bc30c2faaadc
[repository] refactor and optimize '?1' relation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7502
diff
changeset
|
97 |
if not session.is_hook_category_activated('activeintegrity'): |
4687
082e66184f71
[integrity] skip auto-delete for internal sessions (but *not* for super session, see 4642:921737d2e3a8)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4686
diff
changeset
|
98 |
return |
7502
e7190f7e850e
[session] deprecates schema_rproperty in favor of more optimized rtype_eids_rdef which return the rdef (so reusable to gather other data)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7501
diff
changeset
|
99 |
rdef = session.rtype_eids_rdef(rtype, eidfrom, eidto) |
e7190f7e850e
[session] deprecates schema_rproperty in favor of more optimized rtype_eids_rdef which return the rdef (so reusable to gather other data)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7501
diff
changeset
|
100 |
card = rdef.cardinality |
0 | 101 |
# one may be tented to check for neweids but this may cause more than one |
102 |
# relation even with '1?' cardinality if thoses relations are added in the |
|
103 |
# same transaction where the entity is being created. This never occurs from |
|
104 |
# the web interface but may occurs during test or dbapi connection (though |
|
105 |
# not expected for this). So: don't do it, we pretend to ensure repository |
|
106 |
# consistency. |
|
4673
6f8b925a29f4
auto-delete relation with single cardinality using execute, not unsafe_execute, so permissions are correctly checked
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4643
diff
changeset
|
107 |
# |
6189 | 108 |
# notes: |
109 |
# * inlined relations will be implicitly deleted for the subject entity |
|
110 |
# * we don't want read permissions to be applied but we want delete |
|
111 |
# permission to be checked |
|
7503
bc30c2faaadc
[repository] refactor and optimize '?1' relation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7502
diff
changeset
|
112 |
if card[0] in '1?': |
6189 | 113 |
with security_enabled(session, read=False): |
114 |
session.execute('DELETE X %s Y WHERE X eid %%(x)s, ' |
|
115 |
'NOT Y eid %%(y)s' % rtype, |
|
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
116 |
{'x': eidfrom, 'y': eidto}) |
0 | 117 |
if card[1] in '1?': |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
118 |
with security_enabled(session, read=False): |
5711
ea2538a3c622
[repo] fix typo
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5641
diff
changeset
|
119 |
session.execute('DELETE X %s Y WHERE Y eid %%(y)s, ' |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
120 |
'NOT X eid %%(x)s' % rtype, |
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
121 |
{'x': eidfrom, 'y': eidto}) |
1482 | 122 |
|
3694 | 123 |
|
0 | 124 |
class Repository(object): |
125 |
"""a repository provides access to a set of persistent storages for |
|
126 |
entities and relations |
|
127 |
||
128 |
XXX protect pyro access |
|
129 |
""" |
|
1482 | 130 |
|
5442
3ed8afbbdf70
[webconfig] refactor/cleanup debug mode management on startup: simply use config.debugmode instead of debug argument everywhere...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
131 |
def __init__(self, config, vreg=None): |
0 | 132 |
self.config = config |
133 |
if vreg is None: |
|
5442
3ed8afbbdf70
[webconfig] refactor/cleanup debug mode management on startup: simply use config.debugmode instead of debug argument everywhere...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
134 |
vreg = cwvreg.CubicWebVRegistry(config) |
0 | 135 |
self.vreg = vreg |
136 |
self.pyro_registered = False |
|
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
137 |
self.pyro_uri = None |
0 | 138 |
self.info('starting repository from %s', self.config.apphome) |
139 |
# dictionary of opened sessions |
|
140 |
self._sessions = {} |
|
141 |
# list of functions to be called at regular interval |
|
142 |
self._looping_tasks = [] |
|
143 |
# list of running threads |
|
144 |
self._running_threads = [] |
|
145 |
# initial schema, should be build or replaced latter |
|
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
146 |
self.schema = schema.CubicWebSchema(config.appid) |
3240
8604a15995d1
refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3042
diff
changeset
|
147 |
self.vreg.schema = self.schema # until actual schema is loaded... |
5906
d40ced753291
[repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5813
diff
changeset
|
148 |
# shutdown flag |
d40ced753291
[repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5813
diff
changeset
|
149 |
self.shutting_down = False |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
150 |
# sources (additional sources info in the system database) |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
151 |
self.system_source = self.get_source('native', 'system', |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
152 |
config.sources()['system']) |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
153 |
self.sources = [self.system_source] |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
154 |
self.sources_by_uri = {'system': self.system_source} |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
155 |
# querier helper, need to be created after sources initialization |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
156 |
self.querier = querier.QuerierHelper(self, self.schema) |
7543
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
157 |
# cache eid -> (type, physical source, extid, actual source) |
0 | 158 |
self._type_source_cache = {} |
159 |
# cache (extid, source uri) -> eid |
|
160 |
self._extid_cache = {} |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
161 |
# open some connections set |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
162 |
if config.init_cnxset_pool: |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
163 |
self.init_cnxset_pool() |
5079
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
164 |
@onevent('after-registry-reload', self) |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
165 |
def fix_user_classes(self): |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
166 |
usercls = self.vreg['etypes'].etype_class('CWUser') |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
167 |
for session in self._sessions.values(): |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
168 |
if not isinstance(session.user, InternalManager): |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
169 |
session.user.__class__ = usercls |
2959
daabb9bc5233
make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2929
diff
changeset
|
170 |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
171 |
def init_cnxset_pool(self): |
2959
daabb9bc5233
make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2929
diff
changeset
|
172 |
config = self.config |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
173 |
self._cnxsets_pool = Queue.Queue() |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
174 |
self._cnxsets_pool.put_nowait(pool.ConnectionsSet(self.sources)) |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
175 |
if config.quick_start: |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
176 |
# quick start, usually only to get a minimal repository to get cubes |
5084
d8f491cb046c
[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5079
diff
changeset
|
177 |
# information (eg dump/restore/...) |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
178 |
config._cubes = () |
5084
d8f491cb046c
[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5079
diff
changeset
|
179 |
# only load hooks and entity classes in the registry |
7305
1f97c05cfbab
[repository quick start] should modify class attribute to get this actually working (used by a classmethod)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7238
diff
changeset
|
180 |
config.__class__.cube_appobject_path = set(('hooks', 'entities')) |
1f97c05cfbab
[repository quick start] should modify class attribute to get this actually working (used by a classmethod)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7238
diff
changeset
|
181 |
config.__class__.cubicweb_appobject_path = set(('hooks', 'entities')) |
5084
d8f491cb046c
[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5079
diff
changeset
|
182 |
self.set_schema(config.load_schema()) |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
183 |
config['connections-pool-size'] = 1 |
5084
d8f491cb046c
[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5079
diff
changeset
|
184 |
# will be reinitialized later from cubes found in the database |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
185 |
config._cubes = None |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
186 |
elif config.creating: |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
187 |
# repository creation |
0 | 188 |
config.bootstrap_cubes() |
2968
0e3460341023
somewhat painful backport of 3.5 branch, should mostly be ok
Sylvain Thénault <sylvain.thenault@logilab.fr>
diff
changeset
|
189 |
self.set_schema(config.load_schema(), resetvreg=False) |
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
190 |
# need to load the Any and CWUser entity types |
0 | 191 |
etdirectory = join(CW_SOFTWARE_ROOT, 'entities') |
1317 | 192 |
self.vreg.init_registration([etdirectory]) |
3409
c684606a95e0
[bootstrap] metadata hooks need to be registered at bootstrap time for db creation
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
3381
diff
changeset
|
193 |
for modname in ('__init__', 'authobjs', 'wfobjs'): |
c684606a95e0
[bootstrap] metadata hooks need to be registered at bootstrap time for db creation
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
3381
diff
changeset
|
194 |
self.vreg.load_file(join(etdirectory, '%s.py' % modname), |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
195 |
'cubicweb.entities.%s' % modname) |
5447
f29dafba250a
[vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5442
diff
changeset
|
196 |
hooksdirectory = join(CW_SOFTWARE_ROOT, 'hooks') |
f29dafba250a
[vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5442
diff
changeset
|
197 |
self.vreg.load_file(join(hooksdirectory, 'metadata.py'), |
f29dafba250a
[vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5442
diff
changeset
|
198 |
'cubicweb.hooks.metadata') |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
199 |
elif config.read_instance_schema: |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
200 |
# normal start: load the instance schema from the database |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
201 |
self.fill_schema() |
0 | 202 |
else: |
203 |
# test start: use the file system schema (quicker) |
|
2476
1294a6bdf3bf
application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2473
diff
changeset
|
204 |
self.warning("set fs instance'schema") |
0 | 205 |
config.bootstrap_cubes() |
2961 | 206 |
self.set_schema(config.load_schema()) |
0 | 207 |
if not config.creating: |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
208 |
self.init_sources_from_database() |
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
209 |
if 'CWProperty' in self.schema: |
0 | 210 |
self.vreg.init_properties(self.properties()) |
211 |
else: |
|
6128 | 212 |
# call init_creating so that for instance native source can |
213 |
# configurate tsearch according to postgres version |
|
0 | 214 |
for source in self.sources: |
215 |
source.init_creating() |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
216 |
# close initialization connetions set and reopen fresh ones for proper |
0 | 217 |
# initialization now that we know cubes |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
218 |
self._get_cnxset().close(True) |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
219 |
# list of available_cnxsets (we can't iterate on Queue instance) |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
220 |
self.cnxsets = [] |
0 | 221 |
for i in xrange(config['connections-pool-size']): |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
222 |
self.cnxsets.append(pool.ConnectionsSet(self.sources)) |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
223 |
self._cnxsets_pool.put_nowait(self.cnxsets[-1]) |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
224 |
if config.quick_start: |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
225 |
config.init_cubes(self.get_cubes()) |
5093
8d073d2e089d
[optimization] improve massive write performance by optimizing hooks selection
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5090
diff
changeset
|
226 |
self.hm = hook.HooksManager(self.vreg) |
1482 | 227 |
|
0 | 228 |
# internals ############################################################### |
229 |
||
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
230 |
def init_sources_from_database(self): |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
231 |
self.sources_by_eid = {} |
6664
7b6acf314a7a
[repo] don't attempt to read sources from database on quick start. Fix crash after restoring a pre 3.10 database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6629
diff
changeset
|
232 |
if self.config.quick_start \ |
7b6acf314a7a
[repo] don't attempt to read sources from database on quick start. Fix crash after restoring a pre 3.10 database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6629
diff
changeset
|
233 |
or not 'CWSource' in self.schema: # # 3.10 migration |
7550
183a61d1bab9
[3.10] fix typo and error for pre 3.10 migration w/ postgres: we *must* call init_creating else the fti isn't correctly initialized, which may lead to silently rollbacked transaction
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7513
diff
changeset
|
234 |
self.system_source.init_creating() |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
235 |
return |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
236 |
session = self.internal_session() |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
237 |
try: |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
238 |
# FIXME: sources should be ordered (add_entity priority) |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
239 |
for sourceent in session.execute( |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
240 |
'Any S, SN, SA, SC WHERE S is_instance_of CWSource, ' |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
241 |
'S name SN, S type SA, S config SC').entities(): |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
242 |
if sourceent.name == 'system': |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
243 |
self.system_source.eid = sourceent.eid |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
244 |
self.sources_by_eid[sourceent.eid] = self.system_source |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
245 |
self.system_source.init(True, sourceent) |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
246 |
continue |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
247 |
self.add_source(sourceent, add_to_cnxsets=False) |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
248 |
finally: |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
249 |
session.close() |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
250 |
|
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
251 |
def _clear_planning_caches(self): |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
252 |
for cache in ('source_defs', 'is_multi_sources_relation', |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
253 |
'can_cross_relation', 'rel_type_sources'): |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
254 |
clear_cache(self, cache) |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
255 |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
256 |
def add_source(self, sourceent, add_to_cnxsets=True): |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
257 |
source = self.get_source(sourceent.type, sourceent.name, |
6945
28bf94d062a9
[sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6941
diff
changeset
|
258 |
sourceent.host_config, sourceent.eid) |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
259 |
self.sources_by_eid[sourceent.eid] = source |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
260 |
self.sources_by_uri[sourceent.name] = source |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
261 |
if self.config.source_enabled(source): |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
262 |
# call source's init method to complete their initialisation if |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
263 |
# needed (for instance looking for persistent configuration using an |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
264 |
# internal session, which is not possible until connections sets have been |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
265 |
# initialized) |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
266 |
source.init(True, sourceent) |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
267 |
if not source.copy_based_source: |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
268 |
self.sources.append(source) |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
269 |
self.querier.set_planner() |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
270 |
if add_to_cnxsets: |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
271 |
for cnxset in self.cnxsets: |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
272 |
cnxset.add_source(source) |
6724
24bf6f181d0e
[pyro source] store pyro source mapping file into the database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
273 |
else: |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
274 |
source.init(False, sourceent) |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
275 |
self._clear_planning_caches() |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
276 |
|
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
277 |
def remove_source(self, uri): |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
278 |
source = self.sources_by_uri.pop(uri) |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
279 |
del self.sources_by_eid[source.eid] |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
280 |
if self.config.source_enabled(source) and not source.copy_based_source: |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
281 |
self.sources.remove(source) |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
282 |
self.querier.set_planner() |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
283 |
for cnxset in self.cnxsets: |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
284 |
cnxset.remove_source(source) |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
285 |
self._clear_planning_caches() |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
286 |
|
6945
28bf94d062a9
[sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6941
diff
changeset
|
287 |
def get_source(self, type, uri, source_config, eid=None): |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
288 |
# set uri and type in source config so it's available through |
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
289 |
# source_defs() |
0 | 290 |
source_config['uri'] = uri |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
291 |
source_config['type'] = type |
6945
28bf94d062a9
[sources] refactor source creation and options handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6941
diff
changeset
|
292 |
return sources.get_source(type, source_config, self, eid) |
1482 | 293 |
|
2963
12ad88615a12
test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2961
diff
changeset
|
294 |
def set_schema(self, schema, resetvreg=True, rebuildinfered=True): |
12ad88615a12
test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2961
diff
changeset
|
295 |
if rebuildinfered: |
12ad88615a12
test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2961
diff
changeset
|
296 |
schema.rebuild_infered_relations() |
0 | 297 |
self.info('set schema %s %#x', schema.name, id(schema)) |
3240
8604a15995d1
refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3042
diff
changeset
|
298 |
if resetvreg: |
2881
d1a5b77e42bc
fix set_schema/init_cubes order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2880
diff
changeset
|
299 |
if self.config._cubes is None: |
d1a5b77e42bc
fix set_schema/init_cubes order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2880
diff
changeset
|
300 |
self.config.init_cubes(self.get_cubes()) |
5447
f29dafba250a
[vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5442
diff
changeset
|
301 |
# trigger full reload of all appobjects |
3240
8604a15995d1
refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3042
diff
changeset
|
302 |
self.vreg.set_schema(schema) |
8604a15995d1
refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3042
diff
changeset
|
303 |
else: |
8604a15995d1
refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3042
diff
changeset
|
304 |
self.vreg._set_schema(schema) |
0 | 305 |
self.querier.set_schema(schema) |
6126
aca6a2c357fd
[repository] enabled sources refactoring: to avoid error, we should always have all known source in repo.sources_by_uri and only enabled ones in repo.sources, so we still have access to the definition of temporarily disabled sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6061
diff
changeset
|
306 |
# don't use self.sources, we may want to give schema even to disabled |
aca6a2c357fd
[repository] enabled sources refactoring: to avoid error, we should always have all known source in repo.sources_by_uri and only enabled ones in repo.sources, so we still have access to the definition of temporarily disabled sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6061
diff
changeset
|
307 |
# sources |
aca6a2c357fd
[repository] enabled sources refactoring: to avoid error, we should always have all known source in repo.sources_by_uri and only enabled ones in repo.sources, so we still have access to the definition of temporarily disabled sources
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6061
diff
changeset
|
308 |
for source in self.sources_by_uri.values(): |
0 | 309 |
source.set_schema(schema) |
310 |
self.schema = schema |
|
311 |
||
312 |
def fill_schema(self): |
|
313 |
"""lod schema from the repository""" |
|
314 |
from cubicweb.server.schemaserial import deserialize_schema |
|
315 |
self.info('loading schema from the repository') |
|
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
316 |
appschema = schema.CubicWebSchema(self.config.appid) |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
317 |
self.set_schema(self.config.load_bootstrap_schema(), resetvreg=False) |
0 | 318 |
self.debug('deserializing db schema into %s %#x', appschema.name, id(appschema)) |
319 |
session = self.internal_session() |
|
320 |
try: |
|
321 |
try: |
|
322 |
deserialize_schema(appschema, session) |
|
323 |
except BadSchemaDefinition: |
|
324 |
raise |
|
325 |
except Exception, ex: |
|
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
326 |
import traceback |
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
327 |
traceback.print_exc() |
1482 | 328 |
raise Exception('Is the database initialised ? (cause: %s)' % |
0 | 329 |
(ex.args and ex.args[0].strip() or 'unknown')), \ |
330 |
None, sys.exc_info()[-1] |
|
331 |
finally: |
|
332 |
session.close() |
|
333 |
self.set_schema(appschema) |
|
1482 | 334 |
|
0 | 335 |
def start_looping_tasks(self): |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
336 |
if not (self.config.creating or self.config.repairing |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
337 |
or self.config.quick_start): |
4958
665eacdd8c50
[repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4951
diff
changeset
|
338 |
# call instance level initialisation hooks |
665eacdd8c50
[repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4951
diff
changeset
|
339 |
self.hm.call_hooks('server_startup', repo=self) |
665eacdd8c50
[repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4951
diff
changeset
|
340 |
# register a task to cleanup expired session |
6012
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
341 |
self.cleanup_session_time = self.config['cleanup-session-time'] or 60 * 60 * 24 |
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
342 |
assert self.cleanup_session_time > 0 |
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
343 |
cleanup_session_interval = min(60*60, self.cleanup_session_time / 3) |
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
344 |
self.looping_task(cleanup_session_interval, self.clean_sessions) |
0 | 345 |
assert isinstance(self._looping_tasks, list), 'already started' |
2708
60d728bdcba5
allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2667
diff
changeset
|
346 |
for i, (interval, func, args) in enumerate(self._looping_tasks): |
7823
5f145462e041
[repo] ensure we don't restart a task while the repo is shutting down (closes #1942736)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7815
diff
changeset
|
347 |
self._looping_tasks[i] = task = utils.LoopTask(self, interval, func, args) |
0 | 348 |
self.info('starting task %s with interval %.2fs', task.name, |
349 |
interval) |
|
350 |
task.start() |
|
351 |
# ensure no tasks will be further added |
|
352 |
self._looping_tasks = tuple(self._looping_tasks) |
|
353 |
||
2708
60d728bdcba5
allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2667
diff
changeset
|
354 |
def looping_task(self, interval, func, *args): |
0 | 355 |
"""register a function to be called every `interval` seconds. |
1482 | 356 |
|
0 | 357 |
looping tasks can only be registered during repository initialization, |
358 |
once done this method will fail. |
|
359 |
""" |
|
360 |
try: |
|
2708
60d728bdcba5
allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2667
diff
changeset
|
361 |
self._looping_tasks.append( (interval, func, args) ) |
0 | 362 |
except AttributeError: |
363 |
raise RuntimeError("can't add looping task once the repository is started") |
|
364 |
||
365 |
def threaded_task(self, func): |
|
366 |
"""start function in a separated thread""" |
|
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
367 |
t = utils.RepoThread(func, self._running_threads) |
0 | 368 |
t.start() |
1482 | 369 |
|
0 | 370 |
#@locked |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
371 |
def _get_cnxset(self): |
0 | 372 |
try: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
373 |
return self._cnxsets_pool.get(True, timeout=5) |
0 | 374 |
except Queue.Empty: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
375 |
raise Exception('no connections set available after 5 secs, probably either a ' |
4706
6035e96b64dd
added stats for munin collecting #615844 - from 027bbff3659f
arthur
parents:
4689
diff
changeset
|
376 |
'bug in code (too many uncommited/rollbacked ' |
6035e96b64dd
added stats for munin collecting #615844 - from 027bbff3659f
arthur
parents:
4689
diff
changeset
|
377 |
'connections) or too much load on the server (in ' |
0 | 378 |
'which case you can try to set a bigger ' |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
379 |
'connections pool size)') |
1482 | 380 |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
381 |
def _free_cnxset(self, cnxset): |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
382 |
self._cnxsets_pool.put_nowait(cnxset) |
0 | 383 |
|
384 |
def pinfo(self): |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
385 |
# XXX: session.cnxset is accessed from a local storage, would be interesting |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
386 |
# to see if there is a cnxset set in any thread specific data) |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
387 |
return '%s: %s (%s)' % (self._cnxsets_pool.qsize(), |
0 | 388 |
','.join(session.user.login for session in self._sessions.values() |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
389 |
if session.cnxset), |
0 | 390 |
threading.currentThread()) |
391 |
def shutdown(self): |
|
392 |
"""called on server stop event to properly close opened sessions and |
|
393 |
connections |
|
394 |
""" |
|
5906
d40ced753291
[repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5813
diff
changeset
|
395 |
assert not self.shutting_down, 'already shutting down' |
d40ced753291
[repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5813
diff
changeset
|
396 |
self.shutting_down = True |
5749
b4393b681f7a
[repo] on repository shutdown, we've to close the new eid creation connection
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5747
diff
changeset
|
397 |
self.system_source.shutdown() |
0 | 398 |
if isinstance(self._looping_tasks, tuple): # if tasks have been started |
399 |
for looptask in self._looping_tasks: |
|
400 |
self.info('canceling task %s...', looptask.name) |
|
401 |
looptask.cancel() |
|
402 |
looptask.join() |
|
403 |
self.info('task %s finished', looptask.name) |
|
404 |
for thread in self._running_threads: |
|
5376
2c3f14bc2590
[python2.6] don't add a name property on Thread
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5292
diff
changeset
|
405 |
self.info('waiting thread %s...', thread.getName()) |
0 | 406 |
thread.join() |
5376
2c3f14bc2590
[python2.6] don't add a name property on Thread
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5292
diff
changeset
|
407 |
self.info('thread %s finished', thread.getName()) |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
408 |
if not (self.config.creating or self.config.repairing |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
409 |
or self.config.quick_start): |
4946
2654cd5c87e0
[repo] don't call server_shutdown hooks when creating/repairing instance to be consistence with server_startup hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4862
diff
changeset
|
410 |
self.hm.call_hooks('server_shutdown', repo=self) |
0 | 411 |
self.close_sessions() |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
412 |
while not self._cnxsets_pool.empty(): |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
413 |
cnxset = self._cnxsets_pool.get_nowait() |
0 | 414 |
try: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
415 |
cnxset.close(True) |
7815
2a164a9cf81c
[exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7782
diff
changeset
|
416 |
except Exception: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
417 |
self.exception('error while closing %s' % cnxset) |
0 | 418 |
continue |
419 |
if self.pyro_registered: |
|
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
420 |
if self._use_pyrons(): |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
421 |
pyro_unregister(self.config) |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
422 |
self.pyro_uri = None |
0 | 423 |
hits, misses = self.querier.cache_hit, self.querier.cache_miss |
424 |
try: |
|
5747
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
425 |
self.info('rql st cache hit/miss: %s/%s (%s%% hits)', hits, misses, |
0 | 426 |
(hits * 100) / (hits + misses)) |
427 |
hits, misses = self.system_source.cache_hit, self.system_source.cache_miss |
|
428 |
self.info('sql cache hit/miss: %s/%s (%s%% hits)', hits, misses, |
|
429 |
(hits * 100) / (hits + misses)) |
|
430 |
nocache = self.system_source.no_cache |
|
431 |
self.info('sql cache usage: %s/%s (%s%%)', hits+ misses, nocache, |
|
432 |
((hits + misses) * 100) / (hits + misses + nocache)) |
|
433 |
except ZeroDivisionError: |
|
434 |
pass |
|
1482 | 435 |
|
6849
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
436 |
def check_auth_info(self, session, login, authinfo): |
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
437 |
"""validate authentication, raise AuthenticationError on failure, return |
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
438 |
associated CWUser's eid on success. |
0 | 439 |
""" |
440 |
for source in self.sources: |
|
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
441 |
if source.support_entity('CWUser'): |
0 | 442 |
try: |
6849
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
443 |
return source.authenticate(session, login, **authinfo) |
0 | 444 |
except AuthenticationError: |
445 |
continue |
|
446 |
else: |
|
447 |
raise AuthenticationError('authentication failed with all sources') |
|
6849
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
448 |
|
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
449 |
def authenticate_user(self, session, login, **authinfo): |
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
450 |
"""validate login / password, raise AuthenticationError on failure |
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
451 |
return associated CWUser instance on success |
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
452 |
""" |
5a0c2cfc19bf
[repository auth] cleanup email login by turning it into a proper repo-side authentication plugin
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6822
diff
changeset
|
453 |
eid = self.check_auth_info(session, login, authinfo) |
2268
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
454 |
cwuser = self._build_user(session, eid) |
0 | 455 |
if self.config.consider_user_state and \ |
5556
9ab2b4c74baf
[entity] introduce a new 'adapters' registry
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5447
diff
changeset
|
456 |
not cwuser.cw_adapt_to('IWorkflowable').state in cwuser.AUTHENTICABLE_STATES: |
0 | 457 |
raise AuthenticationError('user is not in authenticable state') |
2268
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
458 |
return cwuser |
0 | 459 |
|
460 |
def _build_user(self, session, eid): |
|
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
461 |
"""return a CWUser entity for user with the given eid""" |
2650
18aec79ec3a3
R [vreg] important refactoring of the vregistry, moving behaviour to end dictionnary (and so leaving room for more flexibility ; keep bw compat ; update api usage in cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2647
diff
changeset
|
462 |
cls = self.vreg['etypes'].etype_class('CWUser') |
0 | 463 |
rql = cls.fetch_rql(session.user, ['X eid %(x)s']) |
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
464 |
rset = session.execute(rql, {'x': eid}) |
0 | 465 |
assert len(rset) == 1, rset |
2268
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
466 |
cwuser = rset.get_entity(0, 0) |
6491
ee9a10b6620e
pylint option update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6467
diff
changeset
|
467 |
# pylint: disable=W0104 |
2268
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
468 |
# prefetch / cache cwuser's groups and properties. This is especially |
0 | 469 |
# useful for internal sessions to avoid security insertions |
2268
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
470 |
cwuser.groups |
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
471 |
cwuser.properties |
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
472 |
return cwuser |
1482 | 473 |
|
0 | 474 |
# public (dbapi) interface ################################################ |
1482 | 475 |
|
5587
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
476 |
def stats(self): # XXX restrict to managers session? |
6256
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
477 |
"""Return a dictionary containing some statistics about the repository |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
478 |
resources usage. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
479 |
|
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
480 |
This is a public method, not requiring a session id. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
481 |
""" |
5587
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
482 |
results = {} |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
483 |
querier = self.querier |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
484 |
source = self.system_source |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
485 |
for size, maxsize, hits, misses, title in ( |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
486 |
(len(querier._rql_cache), self.config['rql-cache-size'], |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
487 |
querier.cache_hit, querier.cache_miss, 'rqlt_st'), |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
488 |
(len(source._cache), self.config['rql-cache-size'], |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
489 |
source.cache_hit, source.cache_miss, 'sql'), |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
490 |
): |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
491 |
results['%s_cache_size' % title] = '%s / %s' % (size, maxsize) |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
492 |
results['%s_cache_hit' % title] = hits |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
493 |
results['%s_cache_miss' % title] = misses |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
494 |
results['%s_cache_hit_percent' % title] = (hits * 100) / (hits + misses) |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
495 |
results['sql_no_cache'] = self.system_source.no_cache |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
496 |
results['nb_open_sessions'] = len(self._sessions) |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
497 |
results['nb_active_threads'] = threading.activeCount() |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
498 |
results['looping_tasks'] = ', '.join(str(t) for t in self._looping_tasks) |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
499 |
results['available_cnxsets'] = self._cnxsets_pool.qsize() |
5606
61b28589d33f
merge back to stable some changes made on site for a customer.
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
diff
changeset
|
500 |
results['threads'] = ', '.join(sorted(str(t) for t in threading.enumerate())) |
5587
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
501 |
return results |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
502 |
|
0 | 503 |
def get_schema(self): |
6256
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
504 |
"""Return the instance schema. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
505 |
|
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
506 |
This is a public method, not requiring a session id. |
0 | 507 |
""" |
7723
badfd5524ab6
[repo] Stop setting hashmode on schema
Julien Cristau <julien.cristau@logilab.fr>
parents:
7716
diff
changeset
|
508 |
return self.schema |
0 | 509 |
|
510 |
def get_cubes(self): |
|
6256
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
511 |
"""Return the list of cubes used by this instance. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
512 |
|
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
513 |
This is a public method, not requiring a session id. |
0 | 514 |
""" |
2473
490f88fb99b6
new distinguish repairing/creating from regular start.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2268
diff
changeset
|
515 |
versions = self.get_versions(not (self.config.creating |
4689
4eb1f4490538
[test] skipping versions checking during test is enough, no need for monkey patch
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4687
diff
changeset
|
516 |
or self.config.repairing |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
517 |
or self.config.quick_start |
4689
4eb1f4490538
[test] skipping versions checking during test is enough, no need for monkey patch
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4687
diff
changeset
|
518 |
or self.config.mode == 'test')) |
0 | 519 |
cubes = list(versions) |
520 |
cubes.remove('cubicweb') |
|
521 |
return cubes |
|
522 |
||
6308
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
523 |
def get_option_value(self, option, foreid=None): |
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
524 |
"""Return the value for `option` in the configuration. If `foreid` is |
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
525 |
specified, the actual repository to which this entity belongs is |
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
526 |
derefenced and the option value retrieved from it. |
6257
7eb5f1aed728
[repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6256
diff
changeset
|
527 |
|
7eb5f1aed728
[repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6256
diff
changeset
|
528 |
This is a public method, not requiring a session id. |
7eb5f1aed728
[repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6256
diff
changeset
|
529 |
""" |
7eb5f1aed728
[repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6256
diff
changeset
|
530 |
# XXX we may want to check we don't give sensible information |
6308
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
531 |
if foreid is None: |
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
532 |
return self.config[option] |
7543
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
533 |
_, sourceuri, extid, _ = self.type_and_source_from_eid(foreid) |
6308
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
534 |
if sourceuri == 'system': |
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
535 |
return self.config[option] |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
536 |
cnxset = self._get_cnxset() |
6308
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
537 |
try: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
538 |
cnx = cnxset.connection(sourceuri) |
7064
b8ce2266d026
[repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6889
diff
changeset
|
539 |
# needed to check connection is valid and usable by the current |
b8ce2266d026
[repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6889
diff
changeset
|
540 |
# thread |
b8ce2266d026
[repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6889
diff
changeset
|
541 |
newcnx = self.sources_by_uri[sourceuri].check_connection(cnx) |
b8ce2266d026
[repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6889
diff
changeset
|
542 |
if newcnx is not None: |
b8ce2266d026
[repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6889
diff
changeset
|
543 |
cnx = newcnx |
b8ce2266d026
[repo] fix multi-threads bug with get_option_value: pyro proxy can't be used from a thread to another without caution (closes #1535206)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6889
diff
changeset
|
544 |
return cnx.get_option_value(option, extid) |
6308
c151c730a1ac
[dbapi] get_option_value now has a foreid argument telling the option should be dereferenced to the entity's actual repository (necessary for apycot/local_cache handling)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6257
diff
changeset
|
545 |
finally: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
546 |
self._free_cnxset(cnxset) |
6257
7eb5f1aed728
[repo] new method on repo+dbapi.Connection to get a value from repository's configuration
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6256
diff
changeset
|
547 |
|
0 | 548 |
@cached |
549 |
def get_versions(self, checkversions=False): |
|
6256
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
550 |
"""Return the a dictionary containing cubes used by this instance |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
551 |
as key with their version as value, including cubicweb version. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
552 |
|
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
553 |
This is a public method, not requiring a session id. |
0 | 554 |
""" |
555 |
from logilab.common.changelog import Version |
|
556 |
vcconf = {} |
|
557 |
session = self.internal_session() |
|
558 |
try: |
|
559 |
for pk, version in session.execute( |
|
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
560 |
'Any K,V WHERE P is CWProperty, P value V, P pkey K, ' |
0 | 561 |
'P pkey ~="system.version.%"', build_descr=False): |
562 |
cube = pk.split('.')[-1] |
|
563 |
# XXX cubicweb migration |
|
564 |
if cube in CW_MIGRATION_MAP: |
|
565 |
cube = CW_MIGRATION_MAP[cube] |
|
566 |
version = Version(version) |
|
567 |
vcconf[cube] = version |
|
568 |
if checkversions: |
|
569 |
if cube != 'cubicweb': |
|
570 |
fsversion = self.config.cube_version(cube) |
|
571 |
else: |
|
572 |
fsversion = self.config.cubicweb_version() |
|
573 |
if version < fsversion: |
|
2476
1294a6bdf3bf
application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2473
diff
changeset
|
574 |
msg = ('instance has %s version %s but %s ' |
0 | 575 |
'is installed. Run "cubicweb-ctl upgrade".') |
576 |
raise ExecutionError(msg % (cube, version, fsversion)) |
|
577 |
finally: |
|
578 |
session.close() |
|
579 |
return vcconf |
|
1482 | 580 |
|
0 | 581 |
@cached |
582 |
def source_defs(self): |
|
6256
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
583 |
"""Return the a dictionary containing source uris as value and a |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
584 |
dictionary describing each source as value. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
585 |
|
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
586 |
This is a public method, not requiring a session id. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
587 |
""" |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
588 |
sources = {} |
0 | 589 |
# remove sensitive information |
6427
c8a5ac2d1eaa
[schema / sources] store data sources as cubicweb entities
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6426
diff
changeset
|
590 |
for uri, source in self.sources_by_uri.iteritems(): |
6722
3341521d857b
[repo source] rename attribute so it's much easier to grasp its role
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6674
diff
changeset
|
591 |
sources[uri] = source.public_config |
0 | 592 |
return sources |
593 |
||
594 |
def properties(self): |
|
6256
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
595 |
"""Return a result set containing system wide properties. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
596 |
|
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
597 |
This is a public method, not requiring a session id. |
da77d3f95079
[repo] docstring cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6211
diff
changeset
|
598 |
""" |
0 | 599 |
session = self.internal_session() |
600 |
try: |
|
5747
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
601 |
# don't use session.execute, we don't want rset.req set |
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
602 |
return self.querier.execute(session, 'Any K,V WHERE P is CWProperty,' |
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
603 |
'P pkey K, P value V, NOT P for_user U', |
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
604 |
build_descr=False) |
0 | 605 |
finally: |
606 |
session.close() |
|
607 |
||
5032 | 608 |
# XXX protect this method: anonymous should be allowed and registration |
609 |
# plugged |
|
1372
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
610 |
def register_user(self, login, password, email=None, **kwargs): |
0 | 611 |
"""check a user with the given login exists, if not create it with the |
612 |
given password. This method is designed to be used for anonymous |
|
613 |
registration on public web site. |
|
614 |
""" |
|
615 |
session = self.internal_session() |
|
1372
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
616 |
# for consistency, keep same error as unique check hook (although not required) |
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
617 |
errmsg = session._('the value "%s" is already used, use another one') |
0 | 618 |
try: |
5073
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
619 |
if (session.execute('CWUser X WHERE X login %(login)s', {'login': login}, |
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
620 |
build_descr=False) |
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
621 |
or session.execute('CWUser X WHERE X use_email C, C address %(login)s', |
5073
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
622 |
{'login': login}, build_descr=False)): |
5030
5238d9a8dfee
[form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4984
diff
changeset
|
623 |
qname = role_name('login', 'subject') |
5238d9a8dfee
[form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4984
diff
changeset
|
624 |
raise ValidationError(None, {qname: errmsg % login}) |
0 | 625 |
# we have to create the user |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
626 |
user = self.vreg['etypes'].etype_class('CWUser')(session) |
0 | 627 |
if isinstance(password, unicode): |
628 |
# password should *always* be utf8 encoded |
|
629 |
password = password.encode('UTF8') |
|
630 |
kwargs['login'] = login |
|
631 |
kwargs['upassword'] = password |
|
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
632 |
self.glob_add_entity(session, EditedEntity(user, **kwargs)) |
0 | 633 |
session.execute('SET X in_group G WHERE X eid %(x)s, G name "users"', |
634 |
{'x': user.eid}) |
|
1372
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
635 |
if email or '@' in login: |
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
636 |
d = {'login': login, 'email': email or login} |
5073
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
637 |
if session.execute('EmailAddress X WHERE X address %(email)s', d, |
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
638 |
build_descr=False): |
5030
5238d9a8dfee
[form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4984
diff
changeset
|
639 |
qname = role_name('address', 'subject') |
5238d9a8dfee
[form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4984
diff
changeset
|
640 |
raise ValidationError(None, {qname: errmsg % d['email']}) |
1372
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
641 |
session.execute('INSERT EmailAddress X: X address %(email)s, ' |
5032 | 642 |
'U primary_email X, U use_email X ' |
5073
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
643 |
'WHERE U login %(login)s', d, build_descr=False) |
0 | 644 |
session.commit() |
645 |
finally: |
|
646 |
session.close() |
|
594
76218d42d21f
return success or not on creation of user
Arthur Lutz <arthur.lutz@logilab.fr>
parents:
479
diff
changeset
|
647 |
return True |
1482 | 648 |
|
6390
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
649 |
def find_users(self, fetch_attrs, **query_attrs): |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
650 |
"""yield user attributes for cwusers matching the given query_attrs |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
651 |
(the result set cannot survive this method call) |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
652 |
|
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
653 |
This can be used by low-privileges account (anonymous comes to |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
654 |
mind). |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
655 |
|
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
656 |
`fetch_attrs`: tuple of attributes to be fetched |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
657 |
`query_attrs`: dict of attr/values to restrict the query |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
658 |
""" |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
659 |
assert query_attrs |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
660 |
if not hasattr(self, '_cwuser_attrs'): |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
661 |
cwuser = self.schema['CWUser'] |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
662 |
self._cwuser_attrs = set(str(rschema) |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
663 |
for rschema, _eschema in cwuser.attribute_definitions() |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
664 |
if not rschema.meta) |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
665 |
cwuserattrs = self._cwuser_attrs |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
666 |
for k in chain(fetch_attrs, query_attrs.iterkeys()): |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
667 |
if k not in cwuserattrs: |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
668 |
raise Exception('bad input for find_user') |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
669 |
session = self.internal_session() |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
670 |
try: |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
671 |
varmaker = rqlvar_maker() |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
672 |
vars = [(attr, varmaker.next()) for attr in fetch_attrs] |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
673 |
rql = 'Any %s WHERE X is CWUser, ' % ','.join(var[1] for var in vars) |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
674 |
rql += ','.join('X %s %s' % (var[0], var[1]) for var in vars) + ',' |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
675 |
rset = session.execute(rql + ','.join('X %s %%(%s)s' % (attr, attr) |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
676 |
for attr in query_attrs.iterkeys()), |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
677 |
query_attrs) |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
678 |
return rset.rows |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
679 |
finally: |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
680 |
session.close() |
3766853656d7
[repo] add a find_users method for under-privileged accounts
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
6368
diff
changeset
|
681 |
|
3647
2941f4a0aab9
refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3629
diff
changeset
|
682 |
def connect(self, login, **kwargs): |
0 | 683 |
"""open a connection for a given user |
684 |
||
685 |
base_url may be needed to send mails |
|
686 |
cnxtype indicate if this is a pyro connection or a in-memory connection |
|
1482 | 687 |
|
0 | 688 |
raise `AuthenticationError` if the authentication failed |
689 |
raise `ConnectionError` if we can't open a connection |
|
690 |
""" |
|
691 |
# use an internal connection |
|
692 |
session = self.internal_session() |
|
693 |
# try to get a user object |
|
4089
ff92c7d692bf
typos, api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3890
diff
changeset
|
694 |
cnxprops = kwargs.pop('cnxprops', None) |
0 | 695 |
try: |
3647
2941f4a0aab9
refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3629
diff
changeset
|
696 |
user = self.authenticate_user(session, login, **kwargs) |
0 | 697 |
finally: |
698 |
session.close() |
|
699 |
session = Session(user, self, cnxprops) |
|
3379
9192ba07890d
use .cw_rset instead of rset on appobject classes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3293
diff
changeset
|
700 |
user._cw = user.cw_rset.req = session |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
701 |
user.cw_clear_relation_cache() |
0 | 702 |
self._sessions[session.id] = session |
5638
cea831403e72
log the user name on session opening for easier debugging
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5603
diff
changeset
|
703 |
self.info('opened session %s for user %s', session.id, login) |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
704 |
self.hm.call_hooks('session_open', session) |
0 | 705 |
# commit session at this point in case write operation has been done |
706 |
# during `session_open` hooks |
|
707 |
session.commit() |
|
708 |
return session.id |
|
709 |
||
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
710 |
def execute(self, sessionid, rqlstring, args=None, build_descr=True, |
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
711 |
txid=None): |
0 | 712 |
"""execute a RQL query |
713 |
||
714 |
* rqlstring should be an unicode string or a plain ascii string |
|
715 |
* args the optional parameters used in the query |
|
716 |
* build_descr is a flag indicating if the description should be |
|
717 |
built on select queries |
|
718 |
""" |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
719 |
session = self._get_session(sessionid, setcnxset=True, txid=txid) |
0 | 720 |
try: |
721 |
try: |
|
5747
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
722 |
rset = self.querier.execute(session, rqlstring, args, |
0 | 723 |
build_descr) |
5747
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
724 |
# NOTE: the web front will (re)build it when needed |
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
725 |
# e.g in facets |
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
726 |
# Zeroed to avoid useless overhead with pyro |
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
727 |
rset._rqlst = None |
d6ac0cd30fde
[rset] do not filter rsets with __getstate__, ensure whatever flies with pyro has no .req attribute, also set the ._rqlst to None since it will be reconstructed later on demand
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
5711
diff
changeset
|
728 |
return rset |
0 | 729 |
except (Unauthorized, RQLSyntaxError): |
730 |
raise |
|
731 |
except ValidationError, ex: |
|
732 |
# need ValidationError normalization here so error may pass |
|
733 |
# through pyro |
|
734 |
if hasattr(ex.entity, 'eid'): |
|
735 |
ex.entity = ex.entity.eid # error raised by yams |
|
736 |
args = list(ex.args) |
|
737 |
args[0] = ex.entity |
|
738 |
ex.args = tuple(args) |
|
739 |
raise |
|
7616
8837e4733888
[repository] catch Exception instead of everything
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7573
diff
changeset
|
740 |
except Exception: |
0 | 741 |
# FIXME: check error to catch internal errors |
5167
529861a73ec8
log RQL query in case of unexpected failure (priceless when porting to new database)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5115
diff
changeset
|
742 |
self.exception('unexpected error while executing %s with %s', rqlstring, args) |
0 | 743 |
raise |
744 |
finally: |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
745 |
session.free_cnxset() |
1482 | 746 |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
747 |
def describe(self, sessionid, eid, txid=None): |
7543
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
748 |
"""return a tuple `(type, physical source uri, extid, actual source |
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
749 |
uri)` for the entity of the given `eid` |
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
750 |
""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
751 |
session = self._get_session(sessionid, setcnxset=True, txid=txid) |
0 | 752 |
try: |
753 |
return self.type_and_source_from_eid(eid, session) |
|
754 |
finally: |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
755 |
session.free_cnxset() |
0 | 756 |
|
757 |
def check_session(self, sessionid): |
|
6012
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
758 |
"""raise `BadConnectionId` if the connection is no more valid, else |
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
759 |
return its latest activity timestamp. |
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
760 |
""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
761 |
return self._get_session(sessionid, setcnxset=False).timestamp |
0 | 762 |
|
6013
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
763 |
def get_shared_data(self, sessionid, key, default=None, pop=False, txdata=False): |
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
764 |
"""return value associated to key in the session's data dictionary or |
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
765 |
session's transaction's data if `txdata` is true. |
0 | 766 |
|
6013
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
767 |
If pop is True, value will be removed from the dictionnary. |
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
768 |
|
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
769 |
If key isn't defined in the dictionnary, value specified by the |
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
770 |
`default` argument will be returned. |
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
771 |
""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
772 |
session = self._get_session(sessionid, setcnxset=False) |
6013
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
773 |
return session.get_shared_data(key, default, pop, txdata) |
0 | 774 |
|
6013
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
775 |
def set_shared_data(self, sessionid, key, value, txdata=False): |
0 | 776 |
"""set value associated to `key` in shared data |
777 |
||
6013
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
778 |
if `txdata` is true, the value will be added to the repository session's |
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
779 |
transaction's data which are cleared on commit/rollback of the current |
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
780 |
transaction. |
0 | 781 |
""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
782 |
session = self._get_session(sessionid, setcnxset=False) |
6013
8ca424bc393b
[dbapi] cleanup shared data api: let access to transaction from dbapi, we can write it after all... Also, querydata is better named txdata
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6012
diff
changeset
|
783 |
session.set_shared_data(key, value, txdata) |
0 | 784 |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
785 |
def commit(self, sessionid, txid=None): |
0 | 786 |
"""commit transaction for the session with the given id""" |
787 |
self.debug('begin commit for session %s', sessionid) |
|
788 |
try: |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
789 |
session = self._get_session(sessionid) |
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
790 |
session.set_tx_data(txid) |
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
791 |
return session.commit() |
1482 | 792 |
except (ValidationError, Unauthorized): |
0 | 793 |
raise |
7815
2a164a9cf81c
[exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7782
diff
changeset
|
794 |
except Exception: |
0 | 795 |
self.exception('unexpected error') |
796 |
raise |
|
1482 | 797 |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
798 |
def rollback(self, sessionid, txid=None): |
0 | 799 |
"""commit transaction for the session with the given id""" |
800 |
self.debug('begin rollback for session %s', sessionid) |
|
801 |
try: |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
802 |
session = self._get_session(sessionid) |
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
803 |
session.set_tx_data(txid) |
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
804 |
session.rollback() |
7815
2a164a9cf81c
[exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7782
diff
changeset
|
805 |
except Exception: |
0 | 806 |
self.exception('unexpected error') |
807 |
raise |
|
808 |
||
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
809 |
def close(self, sessionid, txid=None, checkshuttingdown=True): |
0 | 810 |
"""close the session with the given id""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
811 |
session = self._get_session(sessionid, setcnxset=True, txid=txid, |
1939
67e7379edd96
#343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
1883
diff
changeset
|
812 |
checkshuttingdown=checkshuttingdown) |
0 | 813 |
# operation uncommited before close are rollbacked before hook is called |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
814 |
session.rollback(free_cnxset=False) |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
815 |
self.hm.call_hooks('session_close', session) |
0 | 816 |
# commit session at this point in case write operation has been done |
817 |
# during `session_close` hooks |
|
818 |
session.commit() |
|
819 |
session.close() |
|
820 |
del self._sessions[sessionid] |
|
821 |
self.info('closed session %s for user %s', sessionid, session.user.login) |
|
1482 | 822 |
|
0 | 823 |
def user_info(self, sessionid, props=None): |
824 |
"""this method should be used by client to: |
|
825 |
* check session id validity |
|
826 |
* update user information on each user's request (i.e. groups and |
|
827 |
custom properties) |
|
828 |
""" |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
829 |
session = self._get_session(sessionid, setcnxset=False) |
2245
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
830 |
if props is not None: |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
831 |
self.set_session_props(sessionid, props) |
0 | 832 |
user = session.user |
833 |
return user.eid, user.login, user.groups, user.properties |
|
1482 | 834 |
|
2245
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
835 |
def set_session_props(self, sessionid, props): |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
836 |
"""this method should be used by client to: |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
837 |
* check session id validity |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
838 |
* update user information on each user's request (i.e. groups and |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
839 |
custom properties) |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
840 |
""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
841 |
session = self._get_session(sessionid, setcnxset=False) |
2245
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
842 |
for prop, value in props.items(): |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
843 |
session.change_property(prop, value) |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
844 |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
845 |
def undoable_transactions(self, sessionid, ueid=None, txid=None, |
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
846 |
**actionfilters): |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
847 |
"""See :class:`cubicweb.dbapi.Connection.undoable_transactions`""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
848 |
session = self._get_session(sessionid, setcnxset=True, txid=txid) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
849 |
try: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
850 |
return self.system_source.undoable_transactions(session, ueid, |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
851 |
**actionfilters) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
852 |
finally: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
853 |
session.free_cnxset() |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
854 |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
855 |
def transaction_info(self, sessionid, txuuid, txid=None): |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
856 |
"""See :class:`cubicweb.dbapi.Connection.transaction_info`""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
857 |
session = self._get_session(sessionid, setcnxset=True, txid=txid) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
858 |
try: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
859 |
return self.system_source.tx_info(session, txuuid) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
860 |
finally: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
861 |
session.free_cnxset() |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
862 |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
863 |
def transaction_actions(self, sessionid, txuuid, public=True, txid=None): |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
864 |
"""See :class:`cubicweb.dbapi.Connection.transaction_actions`""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
865 |
session = self._get_session(sessionid, setcnxset=True, txid=txid) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
866 |
try: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
867 |
return self.system_source.tx_actions(session, txuuid, public) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
868 |
finally: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
869 |
session.free_cnxset() |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
870 |
|
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
871 |
def undo_transaction(self, sessionid, txuuid, txid=None): |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
872 |
"""See :class:`cubicweb.dbapi.Connection.undo_transaction`""" |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
873 |
session = self._get_session(sessionid, setcnxset=True, txid=txid) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
874 |
try: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
875 |
return self.system_source.undo_transaction(session, txuuid) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
876 |
finally: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
877 |
session.free_cnxset() |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
878 |
|
0 | 879 |
# public (inter-repository) interface ##################################### |
1482 | 880 |
|
0 | 881 |
def entities_modified_since(self, etypes, mtime): |
882 |
"""function designed to be called from an external repository which |
|
883 |
is using this one as a rql source for synchronization, and return a |
|
884 |
3-uple containing : |
|
885 |
* the local date |
|
886 |
* list of (etype, eid) of entities of the given types which have been |
|
887 |
modified since the given timestamp (actually entities whose full text |
|
888 |
index content has changed) |
|
889 |
* list of (etype, eid) of entities of the given types which have been |
|
890 |
deleted since the given timestamp |
|
891 |
""" |
|
892 |
session = self.internal_session() |
|
1016
26387b836099
use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents:
636
diff
changeset
|
893 |
updatetime = datetime.now() |
0 | 894 |
try: |
895 |
modentities, delentities = self.system_source.modified_entities( |
|
896 |
session, etypes, mtime) |
|
897 |
return updatetime, modentities, delentities |
|
898 |
finally: |
|
899 |
session.close() |
|
900 |
||
901 |
# session handling ######################################################## |
|
1482 | 902 |
|
0 | 903 |
def close_sessions(self): |
904 |
"""close every opened sessions""" |
|
905 |
for sessionid in self._sessions.keys(): |
|
906 |
try: |
|
1939
67e7379edd96
#343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
1883
diff
changeset
|
907 |
self.close(sessionid, checkshuttingdown=False) |
7815
2a164a9cf81c
[exceptions] stop catching any exception in various places (closes #1942716)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7782
diff
changeset
|
908 |
except Exception: # XXX BaseException? |
0 | 909 |
self.exception('error while closing session %s' % sessionid) |
910 |
||
911 |
def clean_sessions(self): |
|
912 |
"""close sessions not used since an amount of time specified in the |
|
913 |
configuration |
|
914 |
""" |
|
6012
d56fd78006cd
[session] cleanup session-time / cleanup-session-time...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
915 |
mintime = time() - self.cleanup_session_time |
0 | 916 |
self.debug('cleaning session unused since %s', |
917 |
strftime('%T', localtime(mintime))) |
|
918 |
nbclosed = 0 |
|
919 |
for session in self._sessions.values(): |
|
920 |
if session.timestamp < mintime: |
|
921 |
self.close(session.id) |
|
922 |
nbclosed += 1 |
|
923 |
return nbclosed |
|
1482 | 924 |
|
7706
359bc86d2827
[session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7698
diff
changeset
|
925 |
def internal_session(self, cnxprops=None, safe=False): |
359bc86d2827
[session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7698
diff
changeset
|
926 |
"""return a dbapi like connection/cursor using internal user which have |
359bc86d2827
[session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7698
diff
changeset
|
927 |
every rights on the repository. The `safe` argument is a boolean flag |
359bc86d2827
[session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7698
diff
changeset
|
928 |
telling if integrity hooks should be activated or not. |
359bc86d2827
[session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7698
diff
changeset
|
929 |
|
359bc86d2827
[session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7698
diff
changeset
|
930 |
*YOU HAVE TO* commit/rollback or close (rollback implicitly) the |
359bc86d2827
[session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7698
diff
changeset
|
931 |
session once the job's done, else you'll leak connections set up to the |
359bc86d2827
[session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7698
diff
changeset
|
932 |
time where no one is available, causing irremediable freeze... |
0 | 933 |
""" |
7706
359bc86d2827
[session] safe internal sessions don't deactivate integrity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7698
diff
changeset
|
934 |
session = InternalSession(self, cnxprops, safe) |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
935 |
session.set_cnxset() |
0 | 936 |
return session |
1482 | 937 |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
938 |
def _get_session(self, sessionid, setcnxset=False, txid=None, |
5813
0b250d72fcfa
[transaction w/ separated web/repo processes] the dbapi should explicitly specify a transaction id to avoid confusion when web server / repository run in separated processes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5808
diff
changeset
|
939 |
checkshuttingdown=True): |
0 | 940 |
"""return the user associated to the given session identifier""" |
5906
d40ced753291
[repository] fix so that when repository is shutting down, internal session in transaction are interrupted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5813
diff
changeset
|
941 |
if checkshuttingdown and self.shutting_down: |
7573
c8f8762c986d
[repo, looping task] raise a custom exception when repository is shuting down, avoid looping task to be restarted in such case. Closes #1021276
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7550
diff
changeset
|
942 |
raise ShuttingDown('Repository is shutting down') |
0 | 943 |
try: |
944 |
session = self._sessions[sessionid] |
|
945 |
except KeyError: |
|
946 |
raise BadConnectionId('No such session %s' % sessionid) |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
947 |
if setcnxset: |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
948 |
session.set_tx_data(txid) # must be done before set_cnxset |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
949 |
session.set_cnxset() |
0 | 950 |
return session |
951 |
||
952 |
# data sources handling ################################################### |
|
953 |
# * correspondance between eid and (type, source) |
|
954 |
# * correspondance between eid and local id (i.e. specific to a given source) |
|
1482 | 955 |
|
0 | 956 |
def type_and_source_from_eid(self, eid, session=None): |
7543
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
957 |
"""return a tuple `(type, physical source uri, extid, actual source |
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
958 |
uri)` for the entity of the given `eid` |
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
959 |
""" |
0 | 960 |
try: |
961 |
eid = typed_eid(eid) |
|
962 |
except ValueError: |
|
963 |
raise UnknownEid(eid) |
|
964 |
try: |
|
965 |
return self._type_source_cache[eid] |
|
966 |
except KeyError: |
|
967 |
if session is None: |
|
968 |
session = self.internal_session() |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
969 |
free_cnxset = True |
0 | 970 |
else: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
971 |
free_cnxset = False |
0 | 972 |
try: |
7543
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
973 |
etype, uri, extid, auri = self.system_source.eid_type_source( |
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
974 |
session, eid) |
0 | 975 |
finally: |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
976 |
if free_cnxset: |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
977 |
session.free_cnxset() |
7543
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
978 |
self._type_source_cache[eid] = (etype, uri, extid, auri) |
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
979 |
if uri != 'system': |
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
980 |
self._extid_cache[(extid, uri)] = eid |
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
981 |
return etype, uri, extid, auri |
0 | 982 |
|
983 |
def clear_caches(self, eids): |
|
984 |
etcache = self._type_source_cache |
|
985 |
extidcache = self._extid_cache |
|
986 |
rqlcache = self.querier._rql_cache |
|
987 |
for eid in eids: |
|
988 |
try: |
|
7543
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
989 |
etype, uri, extid, auri = etcache.pop(typed_eid(eid)) # may be a string in some cases |
0 | 990 |
rqlcache.pop('%s X WHERE X eid %s' % (etype, eid), None) |
991 |
extidcache.pop((extid, uri), None) |
|
992 |
except KeyError: |
|
993 |
etype = None |
|
994 |
rqlcache.pop('Any X WHERE X eid %s' % eid, None) |
|
995 |
for source in self.sources: |
|
996 |
source.clear_eid_cache(eid, etype) |
|
1482 | 997 |
|
0 | 998 |
def type_from_eid(self, eid, session=None): |
999 |
"""return the type of the entity with id <eid>""" |
|
1000 |
return self.type_and_source_from_eid(eid, session)[0] |
|
1482 | 1001 |
|
0 | 1002 |
def source_from_eid(self, eid, session=None): |
1003 |
"""return the source for the given entity's eid""" |
|
1004 |
return self.sources_by_uri[self.type_and_source_from_eid(eid, session)[1]] |
|
1482 | 1005 |
|
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1006 |
def querier_cache_key(self, session, rql, args, eidkeys): |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1007 |
cachekey = [rql] |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1008 |
for key in sorted(eidkeys): |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1009 |
try: |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1010 |
etype = self.type_from_eid(args[key], session) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1011 |
except KeyError: |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1012 |
raise QueryError('bad cache key %s (no value)' % key) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1013 |
except TypeError: |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1014 |
raise QueryError('bad cache key %s (value: %r)' % ( |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1015 |
key, args[key])) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1016 |
cachekey.append(etype) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1017 |
# ensure eid is correctly typed in args |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1018 |
args[key] = typed_eid(args[key]) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1019 |
return tuple(cachekey) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
1020 |
|
0 | 1021 |
def eid2extid(self, source, eid, session=None): |
1022 |
"""get local id from an eid""" |
|
7543
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
1023 |
etype, uri, extid, _ = self.type_and_source_from_eid(eid, session) |
0 | 1024 |
if source.uri != uri: |
1025 |
# eid not from the given source |
|
1026 |
raise UnknownEid(eid) |
|
1027 |
return extid |
|
1028 |
||
1954 | 1029 |
def extid2eid(self, source, extid, etype, session=None, insert=True, |
7708
45be3a9debe6
[datafeed] for datafeed source, we don't want commit in extid2eid but explicitly handled by the source. Also, we should use 'safe' internal session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7706
diff
changeset
|
1030 |
complete=True, commit=True, sourceparams=None): |
7399
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1031 |
"""Return eid from a local id. If the eid is a negative integer, that |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1032 |
means the entity is known but has been copied back to the system source |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1033 |
hence should be ignored. |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1034 |
|
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1035 |
If no record is found, ie the entity is not known yet: |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1036 |
|
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1037 |
1. an eid is attributed |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1038 |
|
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1039 |
2. the source's :meth:`before_entity_insertion` method is called to |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1040 |
build the entity instance |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1041 |
|
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1042 |
3. unless source's :attr:`should_call_hooks` tell otherwise, |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1043 |
'before_add_entity' hooks are called |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1044 |
|
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1045 |
4. record is added into the system source |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1046 |
|
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1047 |
5. the source's :meth:`after_entity_insertion` method is called to |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1048 |
complete building of the entity instance |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1049 |
|
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1050 |
6. unless source's :attr:`should_call_hooks` tell otherwise, |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1051 |
'before_add_entity' hooks are called |
972ed1843bd8
[multi-sources] support for moving an entity from an external source (closes #343818)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7398
diff
changeset
|
1052 |
""" |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1053 |
uri = 'system' if source.copy_based_source else source.uri |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1054 |
cachekey = (extid, uri) |
0 | 1055 |
try: |
1056 |
return self._extid_cache[cachekey] |
|
1057 |
except KeyError: |
|
1058 |
pass |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1059 |
free_cnxset = False |
0 | 1060 |
if session is None: |
1061 |
session = self.internal_session() |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1062 |
free_cnxset = True |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1063 |
eid = self.system_source.extid2eid(session, uri, extid) |
0 | 1064 |
if eid is not None: |
1065 |
self._extid_cache[cachekey] = eid |
|
7543
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
1066 |
self._type_source_cache[eid] = (etype, uri, extid, source.uri) |
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1067 |
if free_cnxset: |
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1068 |
session.free_cnxset() |
0 | 1069 |
return eid |
1070 |
if not insert: |
|
1071 |
return |
|
1954 | 1072 |
# no link between extid and eid, create one using an internal session |
0 | 1073 |
# since the current session user may not have required permissions to |
1074 |
# do necessary stuff and we don't want to commit user session. |
|
1075 |
# |
|
450
5e14ea0e81c8
a note for later
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
341
diff
changeset
|
1076 |
# Moreover, even if session is already an internal session but is |
0 | 1077 |
# processing a commit, we have to use another one |
1078 |
if not session.is_internal_session: |
|
1079 |
session = self.internal_session() |
|
7398
26695dd703d8
[repository api] definitly kill usage of word 'pool' to refer to connections set used by a session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1080 |
free_cnxset = True |
0 | 1081 |
try: |
1082 |
eid = self.system_source.create_eid(session) |
|
1083 |
self._extid_cache[cachekey] = eid |
|
7543
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
1084 |
self._type_source_cache[eid] = (etype, uri, extid, source.uri) |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1085 |
entity = source.before_entity_insertion( |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1086 |
session, extid, etype, eid, sourceparams) |
0 | 1087 |
if source.should_call_hooks: |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1088 |
self.hm.call_hooks('before_add_entity', session, entity=entity) |
7698
1c7411535c2d
[datafeed / fti] rather control a 'complete' parameter than setting empty attribute values
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7622
diff
changeset
|
1089 |
self.add_info(session, entity, source, extid, complete=complete) |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1090 |
source.after_entity_insertion(session, extid, entity, sourceparams) |
0 | 1091 |
if source.should_call_hooks: |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1092 |
self.hm.call_hooks('after_add_entity', session, entity=entity) |
7716
c893150b2394
[repo ms] commit/rollback session if it has been created in the method
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7708
diff
changeset
|
1093 |
if commit or free_cnxset: |
7708
45be3a9debe6
[datafeed] for datafeed source, we don't want commit in extid2eid but explicitly handled by the source. Also, we should use 'safe' internal session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7706
diff
changeset
|
1094 |
session.commit(free_cnxset) |
0 | 1095 |
return eid |
7708
45be3a9debe6
[datafeed] for datafeed source, we don't want commit in extid2eid but explicitly handled by the source. Also, we should use 'safe' internal session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7706
diff
changeset
|
1096 |
except Exception: |
7716
c893150b2394
[repo ms] commit/rollback session if it has been created in the method
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7708
diff
changeset
|
1097 |
if commit or free_cnxset: |
7708
45be3a9debe6
[datafeed] for datafeed source, we don't want commit in extid2eid but explicitly handled by the source. Also, we should use 'safe' internal session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7706
diff
changeset
|
1098 |
session.rollback(free_cnxset) |
0 | 1099 |
raise |
1482 | 1100 |
|
0 | 1101 |
def add_info(self, session, entity, source, extid=None, complete=True): |
1102 |
"""add type and source info for an eid into the system table, |
|
1103 |
and index the entity with the full text index |
|
1104 |
""" |
|
1105 |
# begin by inserting eid/type/source/extid into the entities table |
|
6426
541659c39f6a
[hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6390
diff
changeset
|
1106 |
hook.CleanupNewEidsCacheOp.get_instance(session).add_data(entity.eid) |
4806
4f12f59b1a13
[fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4756
diff
changeset
|
1107 |
self.system_source.add_info(session, entity, source, extid, complete) |
1482 | 1108 |
|
6953
260f662be125
[repo, ms] fix pb introduced in 6941:9ed02daa7dbb resulting in missing relation deletes when an entity is deleted
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6945
diff
changeset
|
1109 |
def delete_info(self, session, entity, sourceuri, extid, scleanup=None): |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1110 |
"""called by external source when some entity known by the system source |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1111 |
has been deleted in the external source |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1112 |
""" |
5067
adc2122eed03
[repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5043
diff
changeset
|
1113 |
# mark eid as being deleted in session info and setup cache update |
adc2122eed03
[repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5043
diff
changeset
|
1114 |
# operation |
6426
541659c39f6a
[hook/operation] nicer api to achieve same result as set_operation, as described in #1253630
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6390
diff
changeset
|
1115 |
hook.CleanupDeletedEidsCacheOp.get_instance(session).add_data(entity.eid) |
6626
5c20a005bddc
[pyro source] when cleaning relations of an external entity being cleaned up, only delete local relationsd
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6491
diff
changeset
|
1116 |
self._delete_info(session, entity, sourceuri, extid, scleanup) |
1482 | 1117 |
|
6941
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1118 |
def _delete_info(self, session, entity, sourceuri, extid, scleanup=None): |
0 | 1119 |
"""delete system information on deletion of an entity: |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1120 |
* delete all remaining relations from/to this entity |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1121 |
* call delete info on the system source which will transfer record from |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1122 |
the entities table to the deleted_entities table |
0 | 1123 |
""" |
2641
9c33d98a074e
R [schema hooks] big refactoring / reorganization for clearer code, a few fixes on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2631
diff
changeset
|
1124 |
pendingrtypes = session.transaction_data.get('pendingrtypes', ()) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1125 |
# delete remaining relations: if user can delete the entity, he can |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1126 |
# delete all its relations without security checking |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
1127 |
with security_enabled(session, read=False, write=False): |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1128 |
eid = entity.eid |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1129 |
for rschema, _, role in entity.e_schema.relation_definitions(): |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
1130 |
rtype = rschema.type |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
1131 |
if rtype in schema.VIRTUAL_RTYPES or rtype in pendingrtypes: |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
1132 |
continue |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1133 |
if role == 'subject': |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
1134 |
# don't skip inlined relation so they are regularly |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
1135 |
# deleted and so hooks are correctly called |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1136 |
rql = 'DELETE X %s Y WHERE X eid %%(x)s' % rtype |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
1137 |
else: |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1138 |
rql = 'DELETE Y %s X WHERE X eid %%(x)s' % rtype |
6941
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1139 |
if scleanup is not None: |
6626
5c20a005bddc
[pyro source] when cleaning relations of an external entity being cleaned up, only delete local relationsd
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6491
diff
changeset
|
1140 |
# source cleaning: only delete relations stored locally |
6941
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1141 |
# (here, scleanup |
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1142 |
rql += ', NOT (Y cw_source S, S eid %(seid)s)' |
6674
fd9c76196bf7
[repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6664
diff
changeset
|
1143 |
try: |
6941
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1144 |
session.execute(rql, {'x': eid, 'seid': scleanup}, |
6674
fd9c76196bf7
[repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6664
diff
changeset
|
1145 |
build_descr=False) |
7616
8837e4733888
[repository] catch Exception instead of everything
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7573
diff
changeset
|
1146 |
except Exception: |
6674
fd9c76196bf7
[repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6664
diff
changeset
|
1147 |
self.exception('error while cascading delete for entity %s ' |
fd9c76196bf7
[repository] don't crash when cascading delete a relation, this may let the database in an inconsistent state anyway, so simply log the error
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6664
diff
changeset
|
1148 |
'from %s. RQL: %s', entity, sourceuri, rql) |
7501
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1149 |
self.system_source.delete_info_multi(session, [entity], sourceuri) |
0 | 1150 |
|
7501
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1151 |
def _delete_info_multi(self, session, entities, sourceuri, scleanup=None): |
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1152 |
"""same as _delete_info but accepts a list of entities with |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1153 |
the same etype and belinging to the same source. |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1154 |
""" |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1155 |
pendingrtypes = session.transaction_data.get('pendingrtypes', ()) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1156 |
# delete remaining relations: if user can delete the entity, he can |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1157 |
# delete all its relations without security checking |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1158 |
with security_enabled(session, read=False, write=False): |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1159 |
eids = [_e.eid for _e in entities] |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1160 |
in_eids = ','.join((str(eid) for eid in eids)) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1161 |
for rschema, _, role in entities[0].e_schema.relation_definitions(): |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1162 |
rtype = rschema.type |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1163 |
if rtype in schema.VIRTUAL_RTYPES or rtype in pendingrtypes: |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1164 |
continue |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1165 |
if role == 'subject': |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1166 |
# don't skip inlined relation so they are regularly |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1167 |
# deleted and so hooks are correctly called |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1168 |
rql = 'DELETE X %s Y WHERE X eid IN (%s)' % (rtype, in_eids) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1169 |
else: |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1170 |
rql = 'DELETE Y %s X WHERE X eid IN (%s)' % (rtype, in_eids) |
6941
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1171 |
if scleanup is not None: |
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1172 |
# source cleaning: only delete relations stored locally |
6941
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1173 |
rql += ', NOT (Y cw_source S, S eid %(seid)s)' |
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1174 |
try: |
6941
9ed02daa7dbb
[repo, ms] optimise external source deletion by using source's eid instead of name
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6931
diff
changeset
|
1175 |
session.execute(rql, {'seid': scleanup}, build_descr=False) |
7616
8837e4733888
[repository] catch Exception instead of everything
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7573
diff
changeset
|
1176 |
except Exception: |
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1177 |
self.exception('error while cascading delete for entity %s ' |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1178 |
'from %s. RQL: %s', entities, sourceuri, rql) |
7501
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1179 |
self.system_source.delete_info_multi(session, entities, sourceuri) |
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1180 |
|
0 | 1181 |
def locate_relation_source(self, session, subject, rtype, object): |
1182 |
subjsource = self.source_from_eid(subject, session) |
|
1183 |
objsource = self.source_from_eid(object, session) |
|
3042
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1184 |
if not subjsource is objsource: |
0 | 1185 |
source = self.system_source |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1186 |
if not (subjsource.may_cross_relation(rtype) |
3042
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1187 |
and objsource.may_cross_relation(rtype)): |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1188 |
raise MultiSourcesError( |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1189 |
"relation %s can't be crossed among sources" |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1190 |
% rtype) |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1191 |
elif not subjsource.support_relation(rtype): |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1192 |
source = self.system_source |
0 | 1193 |
else: |
1194 |
source = subjsource |
|
3042
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1195 |
if not source.support_relation(rtype, True): |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1196 |
raise MultiSourcesError( |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1197 |
"source %s doesn't support write of %s relation" |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
1198 |
% (source.uri, rtype)) |
0 | 1199 |
return source |
1482 | 1200 |
|
0 | 1201 |
def locate_etype_source(self, etype): |
1202 |
for source in self.sources: |
|
1203 |
if source.support_entity(etype, 1): |
|
1204 |
return source |
|
1205 |
else: |
|
1206 |
raise ETypeNotSupportedBySources(etype) |
|
1482 | 1207 |
|
5068
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1208 |
def init_entity_caches(self, session, entity, source): |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1209 |
"""add entity to session entities cache and repo's extid cache. |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1210 |
Return entity's ext id if the source isn't the system source. |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1211 |
""" |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1212 |
session.set_entity_cache(entity) |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1213 |
suri = source.uri |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1214 |
if suri == 'system': |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1215 |
extid = None |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1216 |
else: |
6957
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1217 |
if source.copy_based_source: |
ffda12be2e9f
[repository] #1460066: backport datafeed cube as cubicweb source
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6953
diff
changeset
|
1218 |
suri = 'system' |
5068
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1219 |
extid = source.get_extid(entity) |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1220 |
self._extid_cache[(str(extid), suri)] = entity.eid |
7543
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
1221 |
self._type_source_cache[entity.eid] = (entity.__regid__, suri, extid, |
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
1222 |
source.uri) |
5068
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1223 |
return extid |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1224 |
|
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1225 |
def glob_add_entity(self, session, edited): |
0 | 1226 |
"""add an entity to the repository |
1482 | 1227 |
|
0 | 1228 |
the entity eid should originaly be None and a unique eid is assigned to |
1229 |
the entity instance |
|
1230 |
""" |
|
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1231 |
entity = edited.entity |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1232 |
entity._cw_is_saved = False # entity has an eid but is not yet saved |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1233 |
# init edited_attributes before calling before_add_entity hooks |
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1234 |
entity.cw_edited = edited |
0 | 1235 |
eschema = entity.e_schema |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1236 |
source = self.locate_etype_source(entity.__regid__) |
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1237 |
# allocate an eid to the entity before calling hooks |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1238 |
entity.eid = self.system_source.create_eid(session) |
5068
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1239 |
# set caches asap |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1240 |
extid = self.init_entity_caches(session, entity, source) |
2600
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1241 |
if server.DEBUG & server.DBG_REPO: |
6466
43d71dd8a8ec
cleanup and 3.10 updates
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6465
diff
changeset
|
1242 |
print 'ADD entity', self, entity.__regid__, entity.eid, edited |
0 | 1243 |
relations = [] |
7503
bc30c2faaadc
[repository] refactor and optimize '?1' relation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7502
diff
changeset
|
1244 |
prefill_entity_caches(entity, relations) |
2929
51cdfe069e10
fix edited_attributes handling when adding entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2875
diff
changeset
|
1245 |
if source.should_call_hooks: |
2968
0e3460341023
somewhat painful backport of 3.5 branch, should mostly be ok
Sylvain Thénault <sylvain.thenault@logilab.fr>
diff
changeset
|
1246 |
self.hm.call_hooks('before_add_entity', session, entity=entity) |
7503
bc30c2faaadc
[repository] refactor and optimize '?1' relation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7502
diff
changeset
|
1247 |
activintegrity = session.is_hook_category_activated('activeintegrity') |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1248 |
for attr in edited.iterkeys(): |
3689
deb13e88e037
follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3606
diff
changeset
|
1249 |
rschema = eschema.subjrels[attr] |
deb13e88e037
follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3606
diff
changeset
|
1250 |
if not rschema.final: # inlined relation |
7503
bc30c2faaadc
[repository] refactor and optimize '?1' relation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7502
diff
changeset
|
1251 |
value = edited[attr] |
bc30c2faaadc
[repository] refactor and optimize '?1' relation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7502
diff
changeset
|
1252 |
relations.append((attr, value)) |
bc30c2faaadc
[repository] refactor and optimize '?1' relation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7502
diff
changeset
|
1253 |
session.update_rel_cache_add(entity.eid, attr, value) |
bc30c2faaadc
[repository] refactor and optimize '?1' relation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7502
diff
changeset
|
1254 |
rdef = session.rtype_eids_rdef(attr, entity.eid, value) |
bc30c2faaadc
[repository] refactor and optimize '?1' relation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7502
diff
changeset
|
1255 |
if rdef.cardinality[1] in '1?' and activintegrity: |
bc30c2faaadc
[repository] refactor and optimize '?1' relation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7502
diff
changeset
|
1256 |
with security_enabled(session, read=False): |
7507
4c043afb104a
fix failures introduced by recent refactoring
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7503
diff
changeset
|
1257 |
session.execute('DELETE X %s Y WHERE Y eid %%(y)s' % attr, |
7503
bc30c2faaadc
[repository] refactor and optimize '?1' relation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7502
diff
changeset
|
1258 |
{'x': entity.eid, 'y': value}) |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1259 |
edited.set_defaults() |
4843
5f7363416765
fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4835
diff
changeset
|
1260 |
if session.is_hook_category_activated('integrity'): |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1261 |
edited.check(creation=True) |
6211
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1262 |
try: |
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1263 |
source.add_entity(session, entity) |
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1264 |
except UniqueTogetherError, exc: |
6465
6401a9d0b5aa
[architecture] introduce adapter to ease transformation of errors before display to the end user. Use it for UniqueTogetherError first
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6464
diff
changeset
|
1265 |
userhdlr = session.vreg['adapters'].select( |
6401a9d0b5aa
[architecture] introduce adapter to ease transformation of errors before display to the end user. Use it for UniqueTogetherError first
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6464
diff
changeset
|
1266 |
'IUserFriendlyError', session, entity=entity, exc=exc) |
6401a9d0b5aa
[architecture] introduce adapter to ease transformation of errors before display to the end user. Use it for UniqueTogetherError first
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6464
diff
changeset
|
1267 |
userhdlr.raise_user_exception() |
0 | 1268 |
self.add_info(session, entity, source, extid, complete=False) |
6368
f907cc7f2875
[repo] properly mark object as saved once added to its source (code much probably gone during a merge...)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6333
diff
changeset
|
1269 |
edited.saved = entity._cw_is_saved = True |
0 | 1270 |
# trigger after_add_entity after after_add_relation |
1271 |
if source.should_call_hooks: |
|
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1272 |
self.hm.call_hooks('after_add_entity', session, entity=entity) |
0 | 1273 |
# call hooks for inlined relations |
1274 |
for attr, value in relations: |
|
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1275 |
self.hm.call_hooks('before_add_relation', session, |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1276 |
eidfrom=entity.eid, rtype=attr, eidto=value) |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1277 |
self.hm.call_hooks('after_add_relation', session, |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1278 |
eidfrom=entity.eid, rtype=attr, eidto=value) |
0 | 1279 |
return entity.eid |
1482 | 1280 |
|
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1281 |
def glob_update_entity(self, session, edited): |
0 | 1282 |
"""replace an entity in the repository |
1283 |
the type and the eid of an entity must not be changed |
|
1284 |
""" |
|
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1285 |
entity = edited.entity |
2600
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1286 |
if server.DEBUG & server.DBG_REPO: |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1287 |
print 'UPDATE entity', entity.__regid__, entity.eid, \ |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1288 |
entity.cw_attr_cache, edited |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1289 |
hm = self.hm |
0 | 1290 |
eschema = entity.e_schema |
2647
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1291 |
session.set_entity_cache(entity) |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1292 |
orig_edited = getattr(entity, 'cw_edited', None) |
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1293 |
entity.cw_edited = edited |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1294 |
try: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1295 |
only_inline_rels, need_fti_update = True, False |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1296 |
relations = [] |
5292
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1297 |
source = self.source_from_eid(entity.eid, session) |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1298 |
for attr in list(edited): |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1299 |
if attr == 'eid': |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1300 |
continue |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1301 |
rschema = eschema.subjrels[attr] |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1302 |
if rschema.final: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1303 |
if getattr(eschema.rdef(attr), 'fulltextindexed', False): |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1304 |
need_fti_update = True |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1305 |
only_inline_rels = False |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1306 |
else: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1307 |
# inlined relation |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1308 |
previous_value = entity.related(attr) or None |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1309 |
if previous_value is not None: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1310 |
previous_value = previous_value[0][0] # got a result set |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1311 |
if previous_value == entity.cw_attr_cache[attr]: |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1312 |
previous_value = None |
5292
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1313 |
elif source.should_call_hooks: |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1314 |
hm.call_hooks('before_delete_relation', session, |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1315 |
eidfrom=entity.eid, rtype=attr, |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1316 |
eidto=previous_value) |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1317 |
relations.append((attr, edited[attr], previous_value)) |
5292
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1318 |
if source.should_call_hooks: |
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1319 |
# call hooks for inlined relations |
6211
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1320 |
for attr, value, _t in relations: |
5292
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1321 |
hm.call_hooks('before_add_relation', session, |
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1322 |
eidfrom=entity.eid, rtype=attr, eidto=value) |
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1323 |
if not only_inline_rels: |
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1324 |
hm.call_hooks('before_update_entity', session, entity=entity) |
5808
2de32c0c293b
[repo] call entity.check() once before_update hooks has been called, to let them a chance to transform data
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5749
diff
changeset
|
1325 |
if session.is_hook_category_activated('integrity'): |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1326 |
edited.check() |
6211
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1327 |
try: |
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1328 |
source.update_entity(session, entity) |
6225 | 1329 |
edited.saved = True |
6211
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1330 |
except UniqueTogetherError, exc: |
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1331 |
etype, rtypes = exc.args |
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1332 |
problems = {} |
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1333 |
for col in rtypes: |
6225 | 1334 |
problems[col] = session._('violates unique_together constraints (%s)') % (','.join(rtypes)) |
6211
e9d125fd1465
nicer error reporting for unique together constraints
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6189
diff
changeset
|
1335 |
raise ValidationError(entity.eid, problems) |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1336 |
self.system_source.update_info(session, entity, need_fti_update) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1337 |
if source.should_call_hooks: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1338 |
if not only_inline_rels: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1339 |
hm.call_hooks('after_update_entity', session, entity=entity) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1340 |
for attr, value, prevvalue in relations: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1341 |
# if the relation is already cached, update existant cache |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1342 |
relcache = entity.cw_relation_cached(attr, 'subject') |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1343 |
if prevvalue is not None: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1344 |
hm.call_hooks('after_delete_relation', session, |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1345 |
eidfrom=entity.eid, rtype=attr, eidto=prevvalue) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1346 |
if relcache is not None: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1347 |
session.update_rel_cache_del(entity.eid, attr, prevvalue) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1348 |
del_existing_rel_if_needed(session, entity.eid, attr, value) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1349 |
if relcache is not None: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1350 |
session.update_rel_cache_add(entity.eid, attr, value) |
2647
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1351 |
else: |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1352 |
entity.cw_set_relation_cache(attr, 'subject', |
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1353 |
session.eid_rset(value)) |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1354 |
hm.call_hooks('after_add_relation', session, |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1355 |
eidfrom=entity.eid, rtype=attr, eidto=value) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1356 |
finally: |
6142
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1357 |
if orig_edited is not None: |
8bc6eac1fac1
[session] cleanup hook / operation / entity edition api
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6139
diff
changeset
|
1358 |
entity.cw_edited = orig_edited |
0 | 1359 |
|
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1360 |
|
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1361 |
def glob_delete_entities(self, session, eids): |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1362 |
"""delete a list of entities and all related entities from the repository""" |
7501
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1363 |
# mark eids as being deleted in session info and setup cache update |
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1364 |
# operation (register pending eids before actual deletion to avoid |
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1365 |
# multiple call to glob_delete_entities) |
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1366 |
op = hook.CleanupDeletedEidsCacheOp.get_instance(session) |
7507
4c043afb104a
fix failures introduced by recent refactoring
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7503
diff
changeset
|
1367 |
if not isinstance(eids, (set, frozenset)): |
4c043afb104a
fix failures introduced by recent refactoring
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7503
diff
changeset
|
1368 |
warn('[3.13] eids should be given as a set', DeprecationWarning, |
4c043afb104a
fix failures introduced by recent refactoring
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7503
diff
changeset
|
1369 |
stacklevel=2) |
4c043afb104a
fix failures introduced by recent refactoring
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7503
diff
changeset
|
1370 |
eids = frozenset(eids) |
7501
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1371 |
eids = eids - op._container |
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1372 |
op._container |= eids |
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1373 |
data_by_etype_source = {} # values are ([list of eids], |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1374 |
# [list of extid], |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1375 |
# [list of entities]) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1376 |
# |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1377 |
# WARNING: the way this dictionary is populated is heavily optimized |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1378 |
# and does not use setdefault on purpose. Unless a new release |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1379 |
# of the Python interpreter advertises large perf improvements |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1380 |
# in setdefault, this should not be changed without profiling. |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1381 |
|
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1382 |
for eid in eids: |
7543
570522300e22
[ms, entity metas] add 'actual source' to entities table / base entity metadata cache. Closes #1767090
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7514
diff
changeset
|
1383 |
etype, sourceuri, extid, _ = self.type_and_source_from_eid(eid, session) |
7501
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1384 |
# XXX should cache entity's cw_metainformation |
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1385 |
entity = session.entity_from_eid(eid, etype) |
7501
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1386 |
try: |
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1387 |
data_by_etype_source[(etype, sourceuri)].append(entity) |
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1388 |
except KeyError: |
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1389 |
data_by_etype_source[(etype, sourceuri)] = [entity] |
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1390 |
for (etype, sourceuri), entities in data_by_etype_source.iteritems(): |
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1391 |
if server.DEBUG & server.DBG_REPO: |
7501
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1392 |
print 'DELETE entities', etype, [entity.eid for entity in entities] |
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1393 |
source = self.sources_by_uri[sourceuri] |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1394 |
if source.should_call_hooks: |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1395 |
self.hm.call_hooks('before_delete_entity', session, entities=entities) |
7501
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1396 |
self._delete_info_multi(session, entities, sourceuri) |
6889
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1397 |
source.delete_entities(session, entities) |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1398 |
if source.should_call_hooks: |
37668bf302f5
improve massive deletion performance
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
6801
diff
changeset
|
1399 |
self.hm.call_hooks('after_delete_entity', session, entities=entities) |
7501
2983dd24494a
[repository] refactor/cleanup entity deletion methods
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7373
diff
changeset
|
1400 |
# don't clear cache here, it is done in a hook on commit |
1482 | 1401 |
|
0 | 1402 |
def glob_add_relation(self, session, subject, rtype, object): |
1403 |
"""add a relation to the repository""" |
|
7237
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1404 |
self.glob_add_relations(session, {rtype: [(subject, object)]}) |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1405 |
|
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1406 |
def glob_add_relations(self, session, relations): |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1407 |
"""add several relations to the repository |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1408 |
|
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1409 |
relations is a dictionary rtype: [(subj_eid, obj_eid), ...] |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1410 |
""" |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1411 |
sources = {} |
7513
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1412 |
subjects_by_types = {} |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1413 |
objects_by_types = {} |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1414 |
activintegrity = session.is_hook_category_activated('activeintegrity') |
7237
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1415 |
for rtype, eids_subj_obj in relations.iteritems(): |
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1416 |
if server.DEBUG & server.DBG_REPO: |
7513
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1417 |
for subjeid, objeid in relations: |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1418 |
print 'ADD relation', subjeid, rtype, objeid |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1419 |
for subjeid, objeid in eids_subj_obj: |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1420 |
source = self.locate_relation_source(session, subjeid, rtype, objeid) |
7238
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1421 |
if source not in sources: |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1422 |
relations_by_rtype = {} |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1423 |
sources[source] = relations_by_rtype |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1424 |
else: |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1425 |
relations_by_rtype = sources[source] |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1426 |
if rtype in relations_by_rtype: |
7513
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1427 |
relations_by_rtype[rtype].append((subjeid, objeid)) |
7237
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1428 |
else: |
7513
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1429 |
relations_by_rtype[rtype] = [(subjeid, objeid)] |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1430 |
if not activintegrity: |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1431 |
continue |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1432 |
# take care to relation of cardinality '?1', as all eids will |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1433 |
# be inserted later, we've remove duplicated eids since they |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1434 |
# won't be catched by `del_existing_rel_if_needed` |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1435 |
rdef = session.rtype_eids_rdef(rtype, subjeid, objeid) |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1436 |
card = rdef.cardinality |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1437 |
if card[0] in '?1': |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1438 |
with security_enabled(session, read=False): |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1439 |
session.execute('DELETE X %s Y WHERE X eid %%(x)s, ' |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1440 |
'NOT Y eid %%(y)s' % rtype, |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1441 |
{'x': subjeid, 'y': objeid}) |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1442 |
subjects = subjects_by_types.setdefault(rdef, {}) |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1443 |
if subjeid in subjects: |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1444 |
del relations_by_rtype[rtype][subjects[subjeid]] |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1445 |
subjects[subjeid] = len(relations_by_rtype[rtype]) - 1 |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1446 |
continue |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1447 |
subjects[subjeid] = len(relations_by_rtype[rtype]) - 1 |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1448 |
if card[1] in '?1': |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1449 |
with security_enabled(session, read=False): |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1450 |
session.execute('DELETE X %s Y WHERE Y eid %%(y)s, ' |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1451 |
'NOT X eid %%(x)s' % rtype, |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1452 |
{'x': subjeid, 'y': objeid}) |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1453 |
objects = objects_by_types.setdefault(rdef, {}) |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1454 |
if objeid in objects: |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1455 |
del relations_by_rtype[rtype][objects[objeid]] |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1456 |
objects[objeid] = len(relations_by_rtype[rtype]) |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1457 |
continue |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1458 |
objects[objeid] = len(relations_by_rtype[rtype]) |
7238
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1459 |
for source, relations_by_rtype in sources.iteritems(): |
7237
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1460 |
if source.should_call_hooks: |
7238
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1461 |
for rtype, source_relations in relations_by_rtype.iteritems(): |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1462 |
self.hm.call_hooks('before_add_relation', session, |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1463 |
rtype=rtype, eids_from_to=source_relations) |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1464 |
for rtype, source_relations in relations_by_rtype.iteritems(): |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1465 |
source.add_relations(session, rtype, source_relations) |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1466 |
rschema = self.schema.rschema(rtype) |
7513
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1467 |
for subjeid, objeid in source_relations: |
8f4422391e5a
[repo integrity] test and fix glob add relation where several entities are added at once for a relation of 1? cardinality
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
7507
diff
changeset
|
1468 |
session.update_rel_cache_add(subjeid, rtype, objeid, rschema.symmetric) |
7237
9f619715665b
[server] improve the speed of setting relations between entities (closes #1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7134
diff
changeset
|
1469 |
if source.should_call_hooks: |
7238
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1470 |
for rtype, source_relations in relations_by_rtype.iteritems(): |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1471 |
self.hm.call_hooks('after_add_relation', session, |
576abb8c4626
fix implementation of repository.glob_add_relations (closes ##1625257)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7237
diff
changeset
|
1472 |
rtype=rtype, eids_from_to=source_relations) |
0 | 1473 |
|
1474 |
def glob_delete_relation(self, session, subject, rtype, object): |
|
1475 |
"""delete a relation from the repository""" |
|
2600
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1476 |
if server.DEBUG & server.DBG_REPO: |
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1477 |
print 'DELETE relation', subject, rtype, object |
0 | 1478 |
source = self.locate_relation_source(session, subject, rtype, object) |
1479 |
if source.should_call_hooks: |
|
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1480 |
self.hm.call_hooks('before_delete_relation', session, |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1481 |
eidfrom=subject, rtype=rtype, eidto=object) |
0 | 1482 |
source.delete_relation(session, subject, rtype, object) |
2647
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1483 |
rschema = self.schema.rschema(rtype) |
4467
0e73d299730a
fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4252
diff
changeset
|
1484 |
session.update_rel_cache_del(subject, rtype, object, rschema.symmetric) |
0e73d299730a
fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4252
diff
changeset
|
1485 |
if rschema.symmetric: |
0e73d299730a
fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4252
diff
changeset
|
1486 |
# on symmetric relation, we can't now in which sense it's |
0 | 1487 |
# stored so try to delete both |
1488 |
source.delete_relation(session, object, rtype, subject) |
|
1489 |
if source.should_call_hooks: |
|
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1490 |
self.hm.call_hooks('after_delete_relation', session, |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1491 |
eidfrom=subject, rtype=rtype, eidto=object) |
0 | 1492 |
|
1493 |
||
1494 |
# pyro handling ########################################################### |
|
1482 | 1495 |
|
6801
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1496 |
@property |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1497 |
@cached |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1498 |
def pyro_appid(self): |
6061
7cb29eab1c9d
[pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
1499 |
from logilab.common import pyro_ext as pyro |
7cb29eab1c9d
[pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
1500 |
config = self.config |
7cb29eab1c9d
[pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
1501 |
appid = '%s.%s' % pyro.ns_group_and_id( |
7cb29eab1c9d
[pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
1502 |
config['pyro-instance-id'] or config.appid, |
7cb29eab1c9d
[pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
1503 |
config['pyro-ns-group']) |
7cb29eab1c9d
[pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
1504 |
# ensure config['pyro-instance-id'] is a full qualified pyro name |
7cb29eab1c9d
[pyro] instance id cleanup, ensure config['pyro-instance-id'] is a fully qualified pyro id on the server side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5920
diff
changeset
|
1505 |
config['pyro-instance-id'] = appid |
6801
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1506 |
return appid |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1507 |
|
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1508 |
def _use_pyrons(self): |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1509 |
"""return True if the pyro-ns-host is set to something else |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1510 |
than NO_PYRONS, meaning we want to go through a pyro |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1511 |
nameserver""" |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1512 |
return self.config['pyro-ns-host'] != 'NO_PYRONS' |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1513 |
|
6801
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1514 |
def pyro_register(self, host=''): |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1515 |
"""register the repository as a pyro object""" |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1516 |
from logilab.common import pyro_ext as pyro |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1517 |
daemon = pyro.register_object(self, self.pyro_appid, |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1518 |
daemonhost=self.config['pyro-host'], |
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1519 |
nshost=self.config['pyro-ns-host'], |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1520 |
use_pyrons=self._use_pyrons()) |
6801
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1521 |
self.info('repository registered as a pyro object %s', self.pyro_appid) |
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1522 |
self.pyro_uri = pyro.get_object_uri(self.pyro_appid) |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1523 |
self.info('pyro uri is: %s', self.pyro_uri) |
0 | 1524 |
self.pyro_registered = True |
6801
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1525 |
# register a looping task to regularly ensure we're still registered |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1526 |
# into the pyro name server |
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1527 |
if self._use_pyrons(): |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1528 |
self.looping_task(60*10, self._ensure_pyro_ns) |
0 | 1529 |
return daemon |
1482 | 1530 |
|
6801
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1531 |
def _ensure_pyro_ns(self): |
7134
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1532 |
if not self._use_pyrons(): |
01544b6d98fa
allow CW to act as a Pyro Server without registering the server to a Pyro Nameserver (closes #1528533)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7085
diff
changeset
|
1533 |
return |
6801
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1534 |
from logilab.common import pyro_ext as pyro |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1535 |
pyro.ns_reregister(self.pyro_appid, nshost=self.config['pyro-ns-host']) |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1536 |
self.info('repository re-registered as a pyro object %s', |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1537 |
self.pyro_appid) |
33952695295b
[repository, pyro] ensure we're still properly registered into pyro name server in a looping task (closes #1336002)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
6722
diff
changeset
|
1538 |
|
1228
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1539 |
# multi-sources planner helpers ########################################### |
1482 | 1540 |
|
1228
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1541 |
@cached |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1542 |
def rel_type_sources(self, rtype): |
6128 | 1543 |
return tuple([source for source in self.sources |
1544 |
if source.support_relation(rtype) |
|
1545 |
or rtype in source.dont_cross_relations]) |
|
1482 | 1546 |
|
1228
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1547 |
@cached |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1548 |
def can_cross_relation(self, rtype): |
6128 | 1549 |
return tuple([source for source in self.sources |
1550 |
if source.support_relation(rtype) |
|
1551 |
and rtype in source.cross_relations]) |
|
1482 | 1552 |
|
1228
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1553 |
@cached |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1554 |
def is_multi_sources_relation(self, rtype): |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1555 |
return any(source for source in self.sources |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1556 |
if not source is self.system_source |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1557 |
and source.support_relation(rtype)) |
1482 | 1558 |
|
7083
b8e35cde46e9
help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7064
diff
changeset
|
1559 |
# these are overridden by set_log_methods below |
b8e35cde46e9
help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7064
diff
changeset
|
1560 |
# only defining here to prevent pylint from complaining |
b8e35cde46e9
help pylint by explicitely defining some attributes
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
7064
diff
changeset
|
1561 |
info = warning = error = critical = exception = debug = lambda msg,*a,**kw: None |
0 | 1562 |
|
1563 |
def pyro_unregister(config): |
|
1564 |
"""unregister the repository from the pyro name server""" |
|
2665
0c6281487f90
[pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2650
diff
changeset
|
1565 |
from logilab.common.pyro_ext import ns_unregister |
3539
f3b14d052798
[pyro] merge pyro-id / pyro-instance-id options, put all pyro options in the same section of the configuration file
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3250
diff
changeset
|
1566 |
appid = config['pyro-instance-id'] or config.appid |
2665
0c6281487f90
[pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2650
diff
changeset
|
1567 |
ns_unregister(appid, config['pyro-ns-group'], config['pyro-ns-host']) |
0 | 1568 |
|
1569 |
||
1570 |
from logging import getLogger |
|
1571 |
from cubicweb import set_log_methods |
|
1572 |
set_log_methods(Repository, getLogger('cubicweb.repository')) |