author | Sylvain Thénault <sylvain.thenault@logilab.fr> |
Wed, 02 Jun 2010 16:15:05 +0200 | |
changeset 5636 | 8138d9c86ac8 |
parent 5627 | a7e40cccdc9b |
child 5655 | ef903fff826d |
permissions | -rw-r--r-- |
5421
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
1 |
# copyright 2003-2010 LOGILAB S.A. (Paris, FRANCE), all rights reserved. |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
2 |
# contact http://www.logilab.fr/ -- mailto:contact@logilab.fr |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
3 |
# |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
4 |
# This file is part of CubicWeb. |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
5 |
# |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
6 |
# CubicWeb is free software: you can redistribute it and/or modify it under the |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
7 |
# terms of the GNU Lesser General Public License as published by the Free |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
8 |
# Software Foundation, either version 2.1 of the License, or (at your option) |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
9 |
# any later version. |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
10 |
# |
5424
8ecbcbff9777
replace logilab-common by CubicWeb in disclaimer
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5421
diff
changeset
|
11 |
# CubicWeb is distributed in the hope that it will be useful, but WITHOUT |
5421
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
12 |
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
13 |
# FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
14 |
# details. |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
15 |
# |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
16 |
# You should have received a copy of the GNU Lesser General Public License along |
8167de96c523
proper licensing information (LGPL-2.1). Hope I get it right this time.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5376
diff
changeset
|
17 |
# with CubicWeb. If not, see <http://www.gnu.org/licenses/>. |
0 | 18 |
"""Defines the central class for the CubicWeb RQL server: the repository. |
19 |
||
20 |
The repository is an abstraction allowing execution of rql queries against |
|
21 |
data sources. Most of the work is actually done in helper classes. The |
|
22 |
repository mainly: |
|
23 |
||
24 |
* brings these classes all together to provide a single access |
|
2476
1294a6bdf3bf
application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2473
diff
changeset
|
25 |
point to a cubicweb instance. |
0 | 26 |
* handles session management |
27 |
* provides method for pyro registration, to call if pyro is enabled |
|
5587
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
28 |
""" |
0 | 29 |
|
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
30 |
from __future__ import with_statement |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
31 |
|
0 | 32 |
__docformat__ = "restructuredtext en" |
33 |
||
34 |
import sys |
|
5587
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
35 |
import threading |
0 | 36 |
import Queue |
4719
aaed3f813ef8
kill dead/useless code as suggested by pylint
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4714
diff
changeset
|
37 |
from os.path import join |
1016
26387b836099
use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents:
636
diff
changeset
|
38 |
from datetime import datetime |
0 | 39 |
from time import time, localtime, strftime |
40 |
||
41 |
from logilab.common.decorators import cached |
|
3757 | 42 |
from logilab.common.compat import any |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
43 |
from logilab.common import flatten |
0 | 44 |
|
45 |
from yams import BadSchemaDefinition |
|
5030
5238d9a8dfee
[form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4984
diff
changeset
|
46 |
from yams.schema import role_name |
0 | 47 |
from rql import RQLSyntaxError |
48 |
||
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
49 |
from cubicweb import (CW_SOFTWARE_ROOT, CW_MIGRATION_MAP, QueryError, |
2709
6ee816eb9f25
[hooksmanager] hooks should be reseted after vreg reload
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2708
diff
changeset
|
50 |
UnknownEid, AuthenticationError, ExecutionError, |
3042
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
51 |
ETypeNotSupportedBySources, MultiSourcesError, |
0 | 52 |
BadConnectionId, Unauthorized, ValidationError, |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
53 |
RepositoryError, typed_eid, onevent) |
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
54 |
from cubicweb import cwvreg, schema, server |
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
55 |
from cubicweb.server import utils, hook, pool, querier, sources |
5079
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
56 |
from cubicweb.server.session import Session, InternalSession, InternalManager, \ |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
57 |
security_enabled |
0 | 58 |
|
59 |
||
60 |
def del_existing_rel_if_needed(session, eidfrom, rtype, eidto): |
|
61 |
"""delete existing relation when adding a new one if card is 1 or ? |
|
62 |
||
63 |
have to be done once the new relation has been inserted to avoid having |
|
64 |
an entity without a relation for some time |
|
65 |
||
66 |
this kind of behaviour has to be done in the repository so we don't have |
|
67 |
hooks order hazardness |
|
68 |
""" |
|
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
69 |
# skip that for internal session or if integrity explicitly disabled |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
70 |
# |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
71 |
# XXX we should imo rely on the orm to first fetch existing entity if any |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
72 |
# then delete it. |
4756
a2c73b272c9b
do not check for existant entity when relation of single card is added but integrity hooks are deactivated
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4719
diff
changeset
|
73 |
if session.is_internal_session \ |
5090
8c39d2bf58fd
[repo creation] removing existing entities of 'single' cardinality relatino should be considered as 'activeintegrity' hook. Also don't disable that category during repo creation to avoid pb such as two default workflows for one entity types
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5084
diff
changeset
|
74 |
or not session.is_hook_category_activated('activeintegrity'): |
4687
082e66184f71
[integrity] skip auto-delete for internal sessions (but *not* for super session, see 4642:921737d2e3a8)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4686
diff
changeset
|
75 |
return |
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
76 |
card = session.schema_rproperty(rtype, eidfrom, eidto, 'cardinality') |
0 | 77 |
# one may be tented to check for neweids but this may cause more than one |
78 |
# relation even with '1?' cardinality if thoses relations are added in the |
|
79 |
# same transaction where the entity is being created. This never occurs from |
|
80 |
# the web interface but may occurs during test or dbapi connection (though |
|
81 |
# not expected for this). So: don't do it, we pretend to ensure repository |
|
82 |
# consistency. |
|
4673
6f8b925a29f4
auto-delete relation with single cardinality using execute, not unsafe_execute, so permissions are correctly checked
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4643
diff
changeset
|
83 |
# |
4686
c55606a5c4b0
[integrity] temporary fix auto-delete relation (see 4673:6f8b925a29f4 and 4642:921737d2e3a8): we want no read perms, though we want to check delete perms. Should be properly fixed in 3.7 once we've proper security control and [unsafe_]execute behaviour on the repository side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4673
diff
changeset
|
84 |
# XXX we don't want read permissions to be applied but we want delete |
c55606a5c4b0
[integrity] temporary fix auto-delete relation (see 4673:6f8b925a29f4 and 4642:921737d2e3a8): we want no read perms, though we want to check delete perms. Should be properly fixed in 3.7 once we've proper security control and [unsafe_]execute behaviour on the repository side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4673
diff
changeset
|
85 |
# permission to be checked |
c55606a5c4b0
[integrity] temporary fix auto-delete relation (see 4673:6f8b925a29f4 and 4642:921737d2e3a8): we want no read perms, though we want to check delete perms. Should be properly fixed in 3.7 once we've proper security control and [unsafe_]execute behaviour on the repository side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4673
diff
changeset
|
86 |
rschema = session.repo.schema.rschema(rtype) |
0 | 87 |
if card[0] in '1?': |
4643
921737d2e3a8
fix optimisation with super session that may lead to integrity loss
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4517
diff
changeset
|
88 |
if not rschema.inlined: # inlined relations will be implicitly deleted |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
89 |
with security_enabled(session, read=False): |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
90 |
session.execute('DELETE X %s Y WHERE X eid %%(x)s, ' |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
91 |
'NOT Y eid %%(y)s' % rtype, |
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
92 |
{'x': eidfrom, 'y': eidto}) |
0 | 93 |
if card[1] in '1?': |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
94 |
with security_enabled(session, read=False): |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
95 |
session.execute('DELETE X %sY WHERE Y eid %%(y)s, ' |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
96 |
'NOT X eid %%(x)s' % rtype, |
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
97 |
{'x': eidfrom, 'y': eidto}) |
1482 | 98 |
|
3694 | 99 |
|
0 | 100 |
class Repository(object): |
101 |
"""a repository provides access to a set of persistent storages for |
|
102 |
entities and relations |
|
103 |
||
104 |
XXX protect pyro access |
|
105 |
""" |
|
1482 | 106 |
|
5442
3ed8afbbdf70
[webconfig] refactor/cleanup debug mode management on startup: simply use config.debugmode instead of debug argument everywhere...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
107 |
def __init__(self, config, vreg=None): |
0 | 108 |
self.config = config |
109 |
if vreg is None: |
|
5442
3ed8afbbdf70
[webconfig] refactor/cleanup debug mode management on startup: simply use config.debugmode instead of debug argument everywhere...
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
110 |
vreg = cwvreg.CubicWebVRegistry(config) |
0 | 111 |
self.vreg = vreg |
112 |
self.pyro_registered = False |
|
113 |
self.info('starting repository from %s', self.config.apphome) |
|
114 |
# dictionary of opened sessions |
|
115 |
self._sessions = {} |
|
116 |
# list of functions to be called at regular interval |
|
117 |
self._looping_tasks = [] |
|
118 |
# list of running threads |
|
119 |
self._running_threads = [] |
|
120 |
# initial schema, should be build or replaced latter |
|
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
121 |
self.schema = schema.CubicWebSchema(config.appid) |
3240
8604a15995d1
refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3042
diff
changeset
|
122 |
self.vreg.schema = self.schema # until actual schema is loaded... |
0 | 123 |
# querier helper, need to be created after sources initialization |
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
124 |
self.querier = querier.QuerierHelper(self, self.schema) |
0 | 125 |
# sources |
126 |
self.sources = [] |
|
127 |
self.sources_by_uri = {} |
|
128 |
# FIXME: store additional sources info in the system database ? |
|
129 |
# FIXME: sources should be ordered (add_entity priority) |
|
130 |
for uri, source_config in config.sources().items(): |
|
131 |
if uri == 'admin': |
|
132 |
# not an actual source |
|
1482 | 133 |
continue |
0 | 134 |
source = self.get_source(uri, source_config) |
135 |
self.sources_by_uri[uri] = source |
|
136 |
self.sources.append(source) |
|
137 |
self.system_source = self.sources_by_uri['system'] |
|
138 |
# ensure system source is the first one |
|
139 |
self.sources.remove(self.system_source) |
|
140 |
self.sources.insert(0, self.system_source) |
|
141 |
# cache eid -> type / source |
|
142 |
self._type_source_cache = {} |
|
143 |
# cache (extid, source uri) -> eid |
|
144 |
self._extid_cache = {} |
|
145 |
# open some connections pools |
|
2959
daabb9bc5233
make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2929
diff
changeset
|
146 |
if config.open_connections_pools: |
daabb9bc5233
make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2929
diff
changeset
|
147 |
self.open_connections_pools() |
5079
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
148 |
@onevent('after-registry-reload', self) |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
149 |
def fix_user_classes(self): |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
150 |
usercls = self.vreg['etypes'].etype_class('CWUser') |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
151 |
for session in self._sessions.values(): |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
152 |
if not isinstance(session.user, InternalManager): |
e646047f80cb
[reload] on registry reloading, we should change class of users of opened session
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5073
diff
changeset
|
153 |
session.user.__class__ = usercls |
2959
daabb9bc5233
make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2929
diff
changeset
|
154 |
|
daabb9bc5233
make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2929
diff
changeset
|
155 |
def open_connections_pools(self): |
daabb9bc5233
make db-restore command work even with no/corrupted database
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2929
diff
changeset
|
156 |
config = self.config |
0 | 157 |
self._available_pools = Queue.Queue() |
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
158 |
self._available_pools.put_nowait(pool.ConnectionsPool(self.sources)) |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
159 |
if config.quick_start: |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
160 |
# quick start, usually only to get a minimal repository to get cubes |
5084
d8f491cb046c
[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5079
diff
changeset
|
161 |
# information (eg dump/restore/...) |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
162 |
config._cubes = () |
5084
d8f491cb046c
[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5079
diff
changeset
|
163 |
# only load hooks and entity classes in the registry |
d8f491cb046c
[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5079
diff
changeset
|
164 |
config.cube_appobject_path = set(('hooks', 'entities')) |
d8f491cb046c
[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5079
diff
changeset
|
165 |
config.cubicweb_appobject_path = set(('hooks', 'entities')) |
d8f491cb046c
[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5079
diff
changeset
|
166 |
self.set_schema(config.load_schema()) |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
167 |
config['connections-pool-size'] = 1 |
5084
d8f491cb046c
[repo] we want a hooks manager and hooks initialized event on quick start (we trigger events such as server_backup / server_restore for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5079
diff
changeset
|
168 |
# will be reinitialized later from cubes found in the database |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
169 |
config._cubes = None |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
170 |
elif config.creating: |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
171 |
# repository creation |
0 | 172 |
config.bootstrap_cubes() |
2968
0e3460341023
somewhat painful backport of 3.5 branch, should mostly be ok
Sylvain Thénault <sylvain.thenault@logilab.fr>
diff
changeset
|
173 |
self.set_schema(config.load_schema(), resetvreg=False) |
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
174 |
# need to load the Any and CWUser entity types |
0 | 175 |
etdirectory = join(CW_SOFTWARE_ROOT, 'entities') |
1317 | 176 |
self.vreg.init_registration([etdirectory]) |
3409
c684606a95e0
[bootstrap] metadata hooks need to be registered at bootstrap time for db creation
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
3381
diff
changeset
|
177 |
for modname in ('__init__', 'authobjs', 'wfobjs'): |
c684606a95e0
[bootstrap] metadata hooks need to be registered at bootstrap time for db creation
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
3381
diff
changeset
|
178 |
self.vreg.load_file(join(etdirectory, '%s.py' % modname), |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
179 |
'cubicweb.entities.%s' % modname) |
5447
f29dafba250a
[vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5442
diff
changeset
|
180 |
hooksdirectory = join(CW_SOFTWARE_ROOT, 'hooks') |
f29dafba250a
[vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5442
diff
changeset
|
181 |
self.vreg.load_file(join(hooksdirectory, 'metadata.py'), |
f29dafba250a
[vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5442
diff
changeset
|
182 |
'cubicweb.hooks.metadata') |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
183 |
elif config.read_instance_schema: |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
184 |
# normal start: load the instance schema from the database |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
185 |
self.fill_schema() |
0 | 186 |
else: |
187 |
# test start: use the file system schema (quicker) |
|
2476
1294a6bdf3bf
application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2473
diff
changeset
|
188 |
self.warning("set fs instance'schema") |
0 | 189 |
config.bootstrap_cubes() |
2961 | 190 |
self.set_schema(config.load_schema()) |
0 | 191 |
if not config.creating: |
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
192 |
if 'CWProperty' in self.schema: |
0 | 193 |
self.vreg.init_properties(self.properties()) |
194 |
# call source's init method to complete their initialisation if |
|
195 |
# needed (for instance looking for persistent configuration using an |
|
196 |
# internal session, which is not possible until pools have been |
|
197 |
# initialized) |
|
198 |
for source in self.sources: |
|
199 |
source.init() |
|
200 |
else: |
|
201 |
# call init_creating so for instance native source can configurate |
|
202 |
# tsearch according to postgres version |
|
203 |
for source in self.sources: |
|
204 |
source.init_creating() |
|
205 |
# close initialization pool and reopen fresh ones for proper |
|
206 |
# initialization now that we know cubes |
|
1482 | 207 |
self._get_pool().close(True) |
2493
9806571ea790
major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2476
diff
changeset
|
208 |
# list of available pools (we can't iterated on Queue instance) |
9806571ea790
major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2476
diff
changeset
|
209 |
self.pools = [] |
0 | 210 |
for i in xrange(config['connections-pool-size']): |
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
211 |
self.pools.append(pool.ConnectionsPool(self.sources)) |
2493
9806571ea790
major refactoring of database dump/restore:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2476
diff
changeset
|
212 |
self._available_pools.put_nowait(self.pools[-1]) |
1883
011e13d74cfc
shuting -> shutting
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
1880
diff
changeset
|
213 |
self._shutting_down = False |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
214 |
if config.quick_start: |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
215 |
config.init_cubes(self.get_cubes()) |
5093
8d073d2e089d
[optimization] improve massive write performance by optimizing hooks selection
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5090
diff
changeset
|
216 |
self.hm = hook.HooksManager(self.vreg) |
1482 | 217 |
|
0 | 218 |
# internals ############################################################### |
219 |
||
220 |
def get_source(self, uri, source_config): |
|
221 |
source_config['uri'] = uri |
|
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
222 |
return sources.get_source(source_config, self.schema, self) |
1482 | 223 |
|
2963
12ad88615a12
test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2961
diff
changeset
|
224 |
def set_schema(self, schema, resetvreg=True, rebuildinfered=True): |
12ad88615a12
test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2961
diff
changeset
|
225 |
if rebuildinfered: |
12ad88615a12
test and fix migration introducing base classes (w/ regard to yams inheritance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2961
diff
changeset
|
226 |
schema.rebuild_infered_relations() |
0 | 227 |
self.info('set schema %s %#x', schema.name, id(schema)) |
3240
8604a15995d1
refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3042
diff
changeset
|
228 |
if resetvreg: |
2881
d1a5b77e42bc
fix set_schema/init_cubes order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2880
diff
changeset
|
229 |
if self.config._cubes is None: |
d1a5b77e42bc
fix set_schema/init_cubes order
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2880
diff
changeset
|
230 |
self.config.init_cubes(self.get_cubes()) |
5447
f29dafba250a
[vreg repo] shouldn't call init_registration in on hook registry bootstrap, already done before
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5442
diff
changeset
|
231 |
# trigger full reload of all appobjects |
3240
8604a15995d1
refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3042
diff
changeset
|
232 |
self.vreg.set_schema(schema) |
8604a15995d1
refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3042
diff
changeset
|
233 |
else: |
8604a15995d1
refactor so that rql rewriter may be used outside the server. Enhance it to be usable for RRQLExpression as well
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3042
diff
changeset
|
234 |
self.vreg._set_schema(schema) |
0 | 235 |
self.querier.set_schema(schema) |
236 |
for source in self.sources: |
|
237 |
source.set_schema(schema) |
|
238 |
self.schema = schema |
|
239 |
||
240 |
def fill_schema(self): |
|
241 |
"""lod schema from the repository""" |
|
242 |
from cubicweb.server.schemaserial import deserialize_schema |
|
243 |
self.info('loading schema from the repository') |
|
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
244 |
appschema = schema.CubicWebSchema(self.config.appid) |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
245 |
self.set_schema(self.config.load_bootstrap_schema(), resetvreg=False) |
0 | 246 |
self.debug('deserializing db schema into %s %#x', appschema.name, id(appschema)) |
247 |
session = self.internal_session() |
|
248 |
try: |
|
249 |
try: |
|
250 |
deserialize_schema(appschema, session) |
|
251 |
except BadSchemaDefinition: |
|
252 |
raise |
|
253 |
except Exception, ex: |
|
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
254 |
import traceback |
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
255 |
traceback.print_exc() |
1482 | 256 |
raise Exception('Is the database initialised ? (cause: %s)' % |
0 | 257 |
(ex.args and ex.args[0].strip() or 'unknown')), \ |
258 |
None, sys.exc_info()[-1] |
|
259 |
finally: |
|
260 |
session.close() |
|
261 |
self.set_schema(appschema) |
|
1482 | 262 |
|
0 | 263 |
def start_looping_tasks(self): |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
264 |
if not (self.config.creating or self.config.repairing |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
265 |
or self.config.quick_start): |
4958
665eacdd8c50
[repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4951
diff
changeset
|
266 |
# call instance level initialisation hooks |
665eacdd8c50
[repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4951
diff
changeset
|
267 |
self.hm.call_hooks('server_startup', repo=self) |
665eacdd8c50
[repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4951
diff
changeset
|
268 |
# register a task to cleanup expired session |
665eacdd8c50
[repo] call server_startup event in start_looping_task the repo is fully started (registered in pyro for instance)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4951
diff
changeset
|
269 |
self.looping_task(self.config['session-time']/3., self.clean_sessions) |
0 | 270 |
assert isinstance(self._looping_tasks, list), 'already started' |
2708
60d728bdcba5
allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2667
diff
changeset
|
271 |
for i, (interval, func, args) in enumerate(self._looping_tasks): |
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
272 |
self._looping_tasks[i] = task = utils.LoopTask(interval, func, args) |
0 | 273 |
self.info('starting task %s with interval %.2fs', task.name, |
274 |
interval) |
|
275 |
task.start() |
|
276 |
# ensure no tasks will be further added |
|
277 |
self._looping_tasks = tuple(self._looping_tasks) |
|
278 |
||
2708
60d728bdcba5
allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2667
diff
changeset
|
279 |
def looping_task(self, interval, func, *args): |
0 | 280 |
"""register a function to be called every `interval` seconds. |
1482 | 281 |
|
0 | 282 |
looping tasks can only be registered during repository initialization, |
283 |
once done this method will fail. |
|
284 |
""" |
|
285 |
try: |
|
2708
60d728bdcba5
allow to specify arbitrary argument when recording a looping task func
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2667
diff
changeset
|
286 |
self._looping_tasks.append( (interval, func, args) ) |
0 | 287 |
except AttributeError: |
288 |
raise RuntimeError("can't add looping task once the repository is started") |
|
289 |
||
290 |
def threaded_task(self, func): |
|
291 |
"""start function in a separated thread""" |
|
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
292 |
t = utils.RepoThread(func, self._running_threads) |
0 | 293 |
t.start() |
1482 | 294 |
|
0 | 295 |
#@locked |
296 |
def _get_pool(self): |
|
297 |
try: |
|
298 |
return self._available_pools.get(True, timeout=5) |
|
299 |
except Queue.Empty: |
|
300 |
raise Exception('no pool available after 5 secs, probably either a ' |
|
4706
6035e96b64dd
added stats for munin collecting #615844 - from 027bbff3659f
arthur
parents:
4689
diff
changeset
|
301 |
'bug in code (too many uncommited/rollbacked ' |
6035e96b64dd
added stats for munin collecting #615844 - from 027bbff3659f
arthur
parents:
4689
diff
changeset
|
302 |
'connections) or too much load on the server (in ' |
0 | 303 |
'which case you can try to set a bigger ' |
304 |
'connections pools size)') |
|
1482 | 305 |
|
0 | 306 |
def _free_pool(self, pool): |
307 |
self._available_pools.put_nowait(pool) |
|
308 |
||
309 |
def pinfo(self): |
|
310 |
# XXX: session.pool is accessed from a local storage, would be interesting |
|
311 |
# to see if there is a pool set in any thread specific data) |
|
312 |
return '%s: %s (%s)' % (self._available_pools.qsize(), |
|
313 |
','.join(session.user.login for session in self._sessions.values() |
|
314 |
if session.pool), |
|
315 |
threading.currentThread()) |
|
316 |
def shutdown(self): |
|
317 |
"""called on server stop event to properly close opened sessions and |
|
318 |
connections |
|
319 |
""" |
|
4776
3bf51379baee
assert we are not already shutting down
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4756
diff
changeset
|
320 |
assert not self._shutting_down, 'already shutting down' |
1883
011e13d74cfc
shuting -> shutting
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
1880
diff
changeset
|
321 |
self._shutting_down = True |
0 | 322 |
if isinstance(self._looping_tasks, tuple): # if tasks have been started |
323 |
for looptask in self._looping_tasks: |
|
324 |
self.info('canceling task %s...', looptask.name) |
|
325 |
looptask.cancel() |
|
326 |
looptask.join() |
|
327 |
self.info('task %s finished', looptask.name) |
|
328 |
for thread in self._running_threads: |
|
5376
2c3f14bc2590
[python2.6] don't add a name property on Thread
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5292
diff
changeset
|
329 |
self.info('waiting thread %s...', thread.getName()) |
0 | 330 |
thread.join() |
5376
2c3f14bc2590
[python2.6] don't add a name property on Thread
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5292
diff
changeset
|
331 |
self.info('thread %s finished', thread.getName()) |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
332 |
if not (self.config.creating or self.config.repairing |
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
333 |
or self.config.quick_start): |
4946
2654cd5c87e0
[repo] don't call server_shutdown hooks when creating/repairing instance to be consistence with server_startup hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4862
diff
changeset
|
334 |
self.hm.call_hooks('server_shutdown', repo=self) |
0 | 335 |
self.close_sessions() |
336 |
while not self._available_pools.empty(): |
|
337 |
pool = self._available_pools.get_nowait() |
|
338 |
try: |
|
339 |
pool.close(True) |
|
340 |
except: |
|
341 |
self.exception('error while closing %s' % pool) |
|
342 |
continue |
|
343 |
if self.pyro_registered: |
|
344 |
pyro_unregister(self.config) |
|
345 |
hits, misses = self.querier.cache_hit, self.querier.cache_miss |
|
346 |
try: |
|
347 |
self.info('rqlt st cache hit/miss: %s/%s (%s%% hits)', hits, misses, |
|
348 |
(hits * 100) / (hits + misses)) |
|
349 |
hits, misses = self.system_source.cache_hit, self.system_source.cache_miss |
|
350 |
self.info('sql cache hit/miss: %s/%s (%s%% hits)', hits, misses, |
|
351 |
(hits * 100) / (hits + misses)) |
|
352 |
nocache = self.system_source.no_cache |
|
353 |
self.info('sql cache usage: %s/%s (%s%%)', hits+ misses, nocache, |
|
354 |
((hits + misses) * 100) / (hits + misses + nocache)) |
|
355 |
except ZeroDivisionError: |
|
356 |
pass |
|
1482 | 357 |
|
2267
e1d2df3f1091
move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2245
diff
changeset
|
358 |
def _login_from_email(self, login): |
e1d2df3f1091
move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2245
diff
changeset
|
359 |
session = self.internal_session() |
e1d2df3f1091
move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2245
diff
changeset
|
360 |
try: |
e1d2df3f1091
move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2245
diff
changeset
|
361 |
rset = session.execute('Any L WHERE U login L, U primary_email M, ' |
5073
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
362 |
'M address %(login)s', {'login': login}, |
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
363 |
build_descr=False) |
2267
e1d2df3f1091
move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2245
diff
changeset
|
364 |
if rset.rowcount == 1: |
e1d2df3f1091
move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2245
diff
changeset
|
365 |
login = rset[0][0] |
e1d2df3f1091
move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2245
diff
changeset
|
366 |
finally: |
e1d2df3f1091
move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2245
diff
changeset
|
367 |
session.close() |
e1d2df3f1091
move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2245
diff
changeset
|
368 |
return login |
e1d2df3f1091
move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2245
diff
changeset
|
369 |
|
3647
2941f4a0aab9
refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3629
diff
changeset
|
370 |
def authenticate_user(self, session, login, **kwargs): |
0 | 371 |
"""validate login / password, raise AuthenticationError on failure |
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
372 |
return associated CWUser instance on success |
0 | 373 |
""" |
2267
e1d2df3f1091
move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2245
diff
changeset
|
374 |
if self.vreg.config['allow-email-login'] and '@' in login: |
e1d2df3f1091
move login by email functionnality on the repository side to avoid buggy call to internal_session from the web interface side
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2245
diff
changeset
|
375 |
login = self._login_from_email(login) |
0 | 376 |
for source in self.sources: |
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
377 |
if source.support_entity('CWUser'): |
0 | 378 |
try: |
3647
2941f4a0aab9
refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3629
diff
changeset
|
379 |
eid = source.authenticate(session, login, **kwargs) |
0 | 380 |
break |
381 |
except AuthenticationError: |
|
382 |
continue |
|
383 |
else: |
|
384 |
raise AuthenticationError('authentication failed with all sources') |
|
2268
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
385 |
cwuser = self._build_user(session, eid) |
0 | 386 |
if self.config.consider_user_state and \ |
5556
9ab2b4c74baf
[entity] introduce a new 'adapters' registry
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5447
diff
changeset
|
387 |
not cwuser.cw_adapt_to('IWorkflowable').state in cwuser.AUTHENTICABLE_STATES: |
0 | 388 |
raise AuthenticationError('user is not in authenticable state') |
2268
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
389 |
return cwuser |
0 | 390 |
|
391 |
def _build_user(self, session, eid): |
|
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
392 |
"""return a CWUser entity for user with the given eid""" |
2650
18aec79ec3a3
R [vreg] important refactoring of the vregistry, moving behaviour to end dictionnary (and so leaving room for more flexibility ; keep bw compat ; update api usage in cw
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2647
diff
changeset
|
393 |
cls = self.vreg['etypes'].etype_class('CWUser') |
0 | 394 |
rql = cls.fetch_rql(session.user, ['X eid %(x)s']) |
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
395 |
rset = session.execute(rql, {'x': eid}) |
0 | 396 |
assert len(rset) == 1, rset |
2268
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
397 |
cwuser = rset.get_entity(0, 0) |
1138
22f634977c95
make pylint happy, fix some bugs on the way
sylvain.thenault@logilab.fr
parents:
1016
diff
changeset
|
398 |
# pylint: disable-msg=W0104 |
2268
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
399 |
# prefetch / cache cwuser's groups and properties. This is especially |
0 | 400 |
# useful for internal sessions to avoid security insertions |
2268
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
401 |
cwuser.groups |
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
402 |
cwuser.properties |
2f336fd5e040
euser->cwuser
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2267
diff
changeset
|
403 |
return cwuser |
1482 | 404 |
|
0 | 405 |
# public (dbapi) interface ################################################ |
1482 | 406 |
|
5587
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
407 |
def stats(self): # XXX restrict to managers session? |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
408 |
results = {} |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
409 |
querier = self.querier |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
410 |
source = self.system_source |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
411 |
for size, maxsize, hits, misses, title in ( |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
412 |
(len(querier._rql_cache), self.config['rql-cache-size'], |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
413 |
querier.cache_hit, querier.cache_miss, 'rqlt_st'), |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
414 |
(len(source._cache), self.config['rql-cache-size'], |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
415 |
source.cache_hit, source.cache_miss, 'sql'), |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
416 |
): |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
417 |
results['%s_cache_size' % title] = '%s / %s' % (size, maxsize) |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
418 |
results['%s_cache_hit' % title] = hits |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
419 |
results['%s_cache_miss' % title] = misses |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
420 |
results['%s_cache_hit_percent' % title] = (hits * 100) / (hits + misses) |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
421 |
results['sql_no_cache'] = self.system_source.no_cache |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
422 |
results['nb_open_sessions'] = len(self._sessions) |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
423 |
results['nb_active_threads'] = threading.activeCount() |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
424 |
results['looping_tasks'] = ', '.join(str(t) for t in self._looping_tasks) |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
425 |
results['available_pools'] = self._available_pools.qsize() |
5606
61b28589d33f
merge back to stable some changes made on site for a customer.
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
diff
changeset
|
426 |
results['threads'] = ', '.join(sorted(str(t) for t in threading.enumerate())) |
5587
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
427 |
return results |
72679e450f6d
[web] dont attempt to update last login time on ldap users, avoiding spurious tb in logs (closes #914464)
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5426
diff
changeset
|
428 |
|
0 | 429 |
def get_schema(self): |
2476
1294a6bdf3bf
application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2473
diff
changeset
|
430 |
"""return the instance schema. This is a public method, not |
0 | 431 |
requiring a session id |
432 |
""" |
|
433 |
try: |
|
434 |
# necessary to support pickling used by pyro |
|
435 |
self.schema.__hashmode__ = 'pickle' |
|
436 |
return self.schema |
|
437 |
finally: |
|
438 |
self.schema.__hashmode__ = None |
|
439 |
||
440 |
def get_cubes(self): |
|
2476
1294a6bdf3bf
application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2473
diff
changeset
|
441 |
"""return the list of cubes used by this instance. This is a |
0 | 442 |
public method, not requiring a session id. |
443 |
""" |
|
2473
490f88fb99b6
new distinguish repairing/creating from regular start.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2268
diff
changeset
|
444 |
versions = self.get_versions(not (self.config.creating |
4689
4eb1f4490538
[test] skipping versions checking during test is enough, no need for monkey patch
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4687
diff
changeset
|
445 |
or self.config.repairing |
5043
fe52dd3936cf
[repo config] cleanup read_instance_schema / bootstrap_schema / creating mess
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5032
diff
changeset
|
446 |
or self.config.quick_start |
4689
4eb1f4490538
[test] skipping versions checking during test is enough, no need for monkey patch
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4687
diff
changeset
|
447 |
or self.config.mode == 'test')) |
0 | 448 |
cubes = list(versions) |
449 |
cubes.remove('cubicweb') |
|
450 |
return cubes |
|
451 |
||
452 |
@cached |
|
453 |
def get_versions(self, checkversions=False): |
|
2476
1294a6bdf3bf
application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2473
diff
changeset
|
454 |
"""return the a dictionary containing cubes used by this instance |
0 | 455 |
as key with their version as value, including cubicweb version. This is a |
456 |
public method, not requiring a session id. |
|
457 |
""" |
|
458 |
from logilab.common.changelog import Version |
|
459 |
vcconf = {} |
|
460 |
session = self.internal_session() |
|
461 |
try: |
|
462 |
for pk, version in session.execute( |
|
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
463 |
'Any K,V WHERE P is CWProperty, P value V, P pkey K, ' |
0 | 464 |
'P pkey ~="system.version.%"', build_descr=False): |
465 |
cube = pk.split('.')[-1] |
|
466 |
# XXX cubicweb migration |
|
467 |
if cube in CW_MIGRATION_MAP: |
|
468 |
cube = CW_MIGRATION_MAP[cube] |
|
469 |
version = Version(version) |
|
470 |
vcconf[cube] = version |
|
471 |
if checkversions: |
|
472 |
if cube != 'cubicweb': |
|
473 |
fsversion = self.config.cube_version(cube) |
|
474 |
else: |
|
475 |
fsversion = self.config.cubicweb_version() |
|
476 |
if version < fsversion: |
|
2476
1294a6bdf3bf
application -> instance where it makes sense
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2473
diff
changeset
|
477 |
msg = ('instance has %s version %s but %s ' |
0 | 478 |
'is installed. Run "cubicweb-ctl upgrade".') |
479 |
raise ExecutionError(msg % (cube, version, fsversion)) |
|
480 |
finally: |
|
481 |
session.close() |
|
482 |
return vcconf |
|
1482 | 483 |
|
0 | 484 |
@cached |
485 |
def source_defs(self): |
|
486 |
sources = self.config.sources().copy() |
|
487 |
# remove manager information |
|
488 |
sources.pop('admin', None) |
|
489 |
# remove sensitive information |
|
490 |
for uri, sourcedef in sources.iteritems(): |
|
491 |
sourcedef = sourcedef.copy() |
|
492 |
self.sources_by_uri[uri].remove_sensitive_information(sourcedef) |
|
493 |
sources[uri] = sourcedef |
|
494 |
return sources |
|
495 |
||
496 |
def properties(self): |
|
497 |
"""return a result set containing system wide properties""" |
|
498 |
session = self.internal_session() |
|
499 |
try: |
|
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
500 |
return session.execute('Any K,V WHERE P is CWProperty,' |
0 | 501 |
'P pkey K, P value V, NOT P for_user U', |
502 |
build_descr=False) |
|
503 |
finally: |
|
504 |
session.close() |
|
505 |
||
5032 | 506 |
# XXX protect this method: anonymous should be allowed and registration |
507 |
# plugged |
|
1372
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
508 |
def register_user(self, login, password, email=None, **kwargs): |
0 | 509 |
"""check a user with the given login exists, if not create it with the |
510 |
given password. This method is designed to be used for anonymous |
|
511 |
registration on public web site. |
|
512 |
""" |
|
513 |
session = self.internal_session() |
|
1372
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
514 |
# for consistency, keep same error as unique check hook (although not required) |
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
515 |
errmsg = session._('the value "%s" is already used, use another one') |
0 | 516 |
try: |
5073
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
517 |
if (session.execute('CWUser X WHERE X login %(login)s', {'login': login}, |
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
518 |
build_descr=False) |
1398
5fe84a5f7035
rename internal entity types to have CW prefix instead of E
sylvain.thenault@logilab.fr
parents:
1372
diff
changeset
|
519 |
or session.execute('CWUser X WHERE X use_email C, C address %(login)s', |
5073
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
520 |
{'login': login}, build_descr=False)): |
5030
5238d9a8dfee
[form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4984
diff
changeset
|
521 |
qname = role_name('login', 'subject') |
5238d9a8dfee
[form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4984
diff
changeset
|
522 |
raise ValidationError(None, {qname: errmsg % login}) |
0 | 523 |
# we have to create the user |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
524 |
user = self.vreg['etypes'].etype_class('CWUser')(session) |
0 | 525 |
if isinstance(password, unicode): |
526 |
# password should *always* be utf8 encoded |
|
527 |
password = password.encode('UTF8') |
|
528 |
kwargs['login'] = login |
|
529 |
kwargs['upassword'] = password |
|
530 |
user.update(kwargs) |
|
531 |
self.glob_add_entity(session, user) |
|
532 |
session.execute('SET X in_group G WHERE X eid %(x)s, G name "users"', |
|
533 |
{'x': user.eid}) |
|
1372
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
534 |
if email or '@' in login: |
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
535 |
d = {'login': login, 'email': email or login} |
5073
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
536 |
if session.execute('EmailAddress X WHERE X address %(email)s', d, |
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
537 |
build_descr=False): |
5030
5238d9a8dfee
[form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4984
diff
changeset
|
538 |
qname = role_name('address', 'subject') |
5238d9a8dfee
[form] put qualified name on validation error, should fix #784299
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4984
diff
changeset
|
539 |
raise ValidationError(None, {qname: errmsg % d['email']}) |
1372
d4264cd876e1
register_user can now also set an email
Florent <florent@secondweb.fr>
parents:
1320
diff
changeset
|
540 |
session.execute('INSERT EmailAddress X: X address %(email)s, ' |
5032 | 541 |
'U primary_email X, U use_email X ' |
5073
a9697325cffa
[repo] don't need rset description on those queries
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5072
diff
changeset
|
542 |
'WHERE U login %(login)s', d, build_descr=False) |
0 | 543 |
session.commit() |
544 |
finally: |
|
545 |
session.close() |
|
594
76218d42d21f
return success or not on creation of user
Arthur Lutz <arthur.lutz@logilab.fr>
parents:
479
diff
changeset
|
546 |
return True |
1482 | 547 |
|
3647
2941f4a0aab9
refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3629
diff
changeset
|
548 |
def connect(self, login, **kwargs): |
0 | 549 |
"""open a connection for a given user |
550 |
||
551 |
base_url may be needed to send mails |
|
552 |
cnxtype indicate if this is a pyro connection or a in-memory connection |
|
1482 | 553 |
|
0 | 554 |
raise `AuthenticationError` if the authentication failed |
555 |
raise `ConnectionError` if we can't open a connection |
|
556 |
""" |
|
557 |
# use an internal connection |
|
558 |
session = self.internal_session() |
|
559 |
# try to get a user object |
|
4089
ff92c7d692bf
typos, api update
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3890
diff
changeset
|
560 |
cnxprops = kwargs.pop('cnxprops', None) |
0 | 561 |
try: |
3647
2941f4a0aab9
refactor repo authentication to allow pluggable authentifier to login with something else than a password
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3629
diff
changeset
|
562 |
user = self.authenticate_user(session, login, **kwargs) |
0 | 563 |
finally: |
564 |
session.close() |
|
565 |
session = Session(user, self, cnxprops) |
|
3379
9192ba07890d
use .cw_rset instead of rset on appobject classes
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3293
diff
changeset
|
566 |
user._cw = user.cw_rset.req = session |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
567 |
user.cw_clear_relation_cache() |
0 | 568 |
self._sessions[session.id] = session |
569 |
self.info('opened %s', session) |
|
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
570 |
self.hm.call_hooks('session_open', session) |
0 | 571 |
# commit session at this point in case write operation has been done |
572 |
# during `session_open` hooks |
|
573 |
session.commit() |
|
574 |
return session.id |
|
575 |
||
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
576 |
def execute(self, sessionid, rqlstring, args=None, build_descr=True): |
0 | 577 |
"""execute a RQL query |
578 |
||
579 |
* rqlstring should be an unicode string or a plain ascii string |
|
580 |
* args the optional parameters used in the query |
|
581 |
* build_descr is a flag indicating if the description should be |
|
582 |
built on select queries |
|
583 |
""" |
|
584 |
session = self._get_session(sessionid, setpool=True) |
|
585 |
try: |
|
586 |
try: |
|
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
587 |
return self.querier.execute(session, rqlstring, args, |
0 | 588 |
build_descr) |
589 |
except (Unauthorized, RQLSyntaxError): |
|
590 |
raise |
|
591 |
except ValidationError, ex: |
|
592 |
# need ValidationError normalization here so error may pass |
|
593 |
# through pyro |
|
594 |
if hasattr(ex.entity, 'eid'): |
|
595 |
ex.entity = ex.entity.eid # error raised by yams |
|
596 |
args = list(ex.args) |
|
597 |
args[0] = ex.entity |
|
598 |
ex.args = tuple(args) |
|
599 |
raise |
|
600 |
except: |
|
601 |
# FIXME: check error to catch internal errors |
|
5167
529861a73ec8
log RQL query in case of unexpected failure (priceless when porting to new database)
Alexandre Fayolle <alexandre.fayolle@logilab.fr>
parents:
5115
diff
changeset
|
602 |
self.exception('unexpected error while executing %s with %s', rqlstring, args) |
0 | 603 |
raise |
604 |
finally: |
|
605 |
session.reset_pool() |
|
1482 | 606 |
|
0 | 607 |
def describe(self, sessionid, eid): |
608 |
"""return a tuple (type, source, extid) for the entity with id <eid>""" |
|
609 |
session = self._get_session(sessionid, setpool=True) |
|
610 |
try: |
|
611 |
return self.type_and_source_from_eid(eid, session) |
|
612 |
finally: |
|
613 |
session.reset_pool() |
|
614 |
||
615 |
def check_session(self, sessionid): |
|
5032 | 616 |
"""raise `BadConnectionId` if the connection is no more valid""" |
0 | 617 |
self._get_session(sessionid, setpool=False) |
618 |
||
619 |
def get_shared_data(self, sessionid, key, default=None, pop=False): |
|
620 |
"""return the session's data dictionary""" |
|
621 |
session = self._get_session(sessionid, setpool=False) |
|
622 |
return session.get_shared_data(key, default, pop) |
|
623 |
||
624 |
def set_shared_data(self, sessionid, key, value, querydata=False): |
|
625 |
"""set value associated to `key` in shared data |
|
626 |
||
627 |
if `querydata` is true, the value will be added to the repository |
|
628 |
session's query data which are cleared on commit/rollback of the current |
|
629 |
transaction, and won't be available through the connexion, only on the |
|
630 |
repository side. |
|
631 |
""" |
|
632 |
session = self._get_session(sessionid, setpool=False) |
|
633 |
session.set_shared_data(key, value, querydata) |
|
634 |
||
635 |
def commit(self, sessionid): |
|
636 |
"""commit transaction for the session with the given id""" |
|
637 |
self.debug('begin commit for session %s', sessionid) |
|
638 |
try: |
|
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
639 |
return self._get_session(sessionid).commit() |
1482 | 640 |
except (ValidationError, Unauthorized): |
0 | 641 |
raise |
642 |
except: |
|
643 |
self.exception('unexpected error') |
|
644 |
raise |
|
1482 | 645 |
|
0 | 646 |
def rollback(self, sessionid): |
647 |
"""commit transaction for the session with the given id""" |
|
648 |
self.debug('begin rollback for session %s', sessionid) |
|
649 |
try: |
|
1880
293fe4b49e28
two in one: #343320: Logging out while deleting a CWUser blocks the cw server / #342692: ensure transaction state when Ctrl-C or other stop signal is received
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
1664
diff
changeset
|
650 |
self._get_session(sessionid).rollback() |
0 | 651 |
except: |
652 |
self.exception('unexpected error') |
|
653 |
raise |
|
654 |
||
1939
67e7379edd96
#343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
1883
diff
changeset
|
655 |
def close(self, sessionid, checkshuttingdown=True): |
0 | 656 |
"""close the session with the given id""" |
1939
67e7379edd96
#343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
1883
diff
changeset
|
657 |
session = self._get_session(sessionid, setpool=True, |
67e7379edd96
#343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
1883
diff
changeset
|
658 |
checkshuttingdown=checkshuttingdown) |
0 | 659 |
# operation uncommited before close are rollbacked before hook is called |
660 |
session.rollback() |
|
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
661 |
self.hm.call_hooks('session_close', session) |
0 | 662 |
# commit session at this point in case write operation has been done |
663 |
# during `session_close` hooks |
|
664 |
session.commit() |
|
665 |
session.close() |
|
666 |
del self._sessions[sessionid] |
|
667 |
self.info('closed session %s for user %s', sessionid, session.user.login) |
|
1482 | 668 |
|
0 | 669 |
def user_info(self, sessionid, props=None): |
670 |
"""this method should be used by client to: |
|
671 |
* check session id validity |
|
672 |
* update user information on each user's request (i.e. groups and |
|
673 |
custom properties) |
|
674 |
""" |
|
675 |
session = self._get_session(sessionid, setpool=False) |
|
2245
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
676 |
if props is not None: |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
677 |
self.set_session_props(sessionid, props) |
0 | 678 |
user = session.user |
679 |
return user.eid, user.login, user.groups, user.properties |
|
1482 | 680 |
|
2245
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
681 |
def set_session_props(self, sessionid, props): |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
682 |
"""this method should be used by client to: |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
683 |
* check session id validity |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
684 |
* update user information on each user's request (i.e. groups and |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
685 |
custom properties) |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
686 |
""" |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
687 |
session = self._get_session(sessionid, setpool=False) |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
688 |
for prop, value in props.items(): |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
689 |
session.change_property(prop, value) |
7463e1a748dd
new set_session_props method exposed by the repository, use it to be sure session language is in sync the request language
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2192
diff
changeset
|
690 |
|
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
691 |
def undoable_transactions(self, sessionid, ueid=None, **actionfilters): |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
692 |
"""See :class:`cubicweb.dbapi.Connection.undoable_transactions`""" |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
693 |
session = self._get_session(sessionid, setpool=True) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
694 |
try: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
695 |
return self.system_source.undoable_transactions(session, ueid, |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
696 |
**actionfilters) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
697 |
finally: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
698 |
session.reset_pool() |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
699 |
|
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
700 |
def transaction_info(self, sessionid, txuuid): |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
701 |
"""See :class:`cubicweb.dbapi.Connection.transaction_info`""" |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
702 |
session = self._get_session(sessionid, setpool=True) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
703 |
try: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
704 |
return self.system_source.tx_info(session, txuuid) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
705 |
finally: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
706 |
session.reset_pool() |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
707 |
|
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
708 |
def transaction_actions(self, sessionid, txuuid, public=True): |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
709 |
"""See :class:`cubicweb.dbapi.Connection.transaction_actions`""" |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
710 |
session = self._get_session(sessionid, setpool=True) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
711 |
try: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
712 |
return self.system_source.tx_actions(session, txuuid, public) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
713 |
finally: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
714 |
session.reset_pool() |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
715 |
|
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
716 |
def undo_transaction(self, sessionid, txuuid): |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
717 |
"""See :class:`cubicweb.dbapi.Connection.undo_transaction`""" |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
718 |
session = self._get_session(sessionid, setpool=True) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
719 |
try: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
720 |
return self.system_source.undo_transaction(session, txuuid) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
721 |
finally: |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
722 |
session.reset_pool() |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
723 |
|
0 | 724 |
# public (inter-repository) interface ##################################### |
1482 | 725 |
|
0 | 726 |
def entities_modified_since(self, etypes, mtime): |
727 |
"""function designed to be called from an external repository which |
|
728 |
is using this one as a rql source for synchronization, and return a |
|
729 |
3-uple containing : |
|
730 |
* the local date |
|
731 |
* list of (etype, eid) of entities of the given types which have been |
|
732 |
modified since the given timestamp (actually entities whose full text |
|
733 |
index content has changed) |
|
734 |
* list of (etype, eid) of entities of the given types which have been |
|
735 |
deleted since the given timestamp |
|
736 |
""" |
|
737 |
session = self.internal_session() |
|
1016
26387b836099
use datetime instead of mx.DateTime
sylvain.thenault@logilab.fr
parents:
636
diff
changeset
|
738 |
updatetime = datetime.now() |
0 | 739 |
try: |
740 |
modentities, delentities = self.system_source.modified_entities( |
|
741 |
session, etypes, mtime) |
|
742 |
return updatetime, modentities, delentities |
|
743 |
finally: |
|
744 |
session.close() |
|
745 |
||
746 |
# session handling ######################################################## |
|
1482 | 747 |
|
0 | 748 |
def close_sessions(self): |
749 |
"""close every opened sessions""" |
|
750 |
for sessionid in self._sessions.keys(): |
|
751 |
try: |
|
1939
67e7379edd96
#343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
1883
diff
changeset
|
752 |
self.close(sessionid, checkshuttingdown=False) |
0 | 753 |
except: |
754 |
self.exception('error while closing session %s' % sessionid) |
|
755 |
||
756 |
def clean_sessions(self): |
|
757 |
"""close sessions not used since an amount of time specified in the |
|
758 |
configuration |
|
759 |
""" |
|
760 |
mintime = time() - self.config['session-time'] |
|
761 |
self.debug('cleaning session unused since %s', |
|
762 |
strftime('%T', localtime(mintime))) |
|
763 |
nbclosed = 0 |
|
764 |
for session in self._sessions.values(): |
|
765 |
if session.timestamp < mintime: |
|
766 |
self.close(session.id) |
|
767 |
nbclosed += 1 |
|
768 |
return nbclosed |
|
1482 | 769 |
|
0 | 770 |
def internal_session(self, cnxprops=None): |
771 |
"""return a dbapi like connection/cursor using internal user which |
|
772 |
have every rights on the repository. You'll *have to* commit/rollback |
|
773 |
or close (rollback implicitly) the session once the job's done, else |
|
774 |
you'll leak connections pool up to the time where no more pool is |
|
775 |
available, causing irremediable freeze... |
|
776 |
""" |
|
777 |
session = InternalSession(self, cnxprops) |
|
778 |
session.set_pool() |
|
779 |
return session |
|
1482 | 780 |
|
1939
67e7379edd96
#343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
1883
diff
changeset
|
781 |
def _get_session(self, sessionid, setpool=False, checkshuttingdown=True): |
0 | 782 |
"""return the user associated to the given session identifier""" |
1939
67e7379edd96
#343379: disturbing message on upgrade
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
1883
diff
changeset
|
783 |
if checkshuttingdown and self._shutting_down: |
1883
011e13d74cfc
shuting -> shutting
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
1880
diff
changeset
|
784 |
raise Exception('Repository is shutting down') |
0 | 785 |
try: |
786 |
session = self._sessions[sessionid] |
|
787 |
except KeyError: |
|
788 |
raise BadConnectionId('No such session %s' % sessionid) |
|
789 |
if setpool: |
|
790 |
session.set_pool() |
|
791 |
return session |
|
792 |
||
793 |
# data sources handling ################################################### |
|
794 |
# * correspondance between eid and (type, source) |
|
795 |
# * correspondance between eid and local id (i.e. specific to a given source) |
|
1482 | 796 |
|
0 | 797 |
def type_and_source_from_eid(self, eid, session=None): |
798 |
"""return a tuple (type, source, extid) for the entity with id <eid>""" |
|
799 |
try: |
|
800 |
eid = typed_eid(eid) |
|
801 |
except ValueError: |
|
802 |
raise UnknownEid(eid) |
|
803 |
try: |
|
804 |
return self._type_source_cache[eid] |
|
805 |
except KeyError: |
|
806 |
if session is None: |
|
807 |
session = self.internal_session() |
|
808 |
reset_pool = True |
|
809 |
else: |
|
810 |
reset_pool = False |
|
811 |
try: |
|
812 |
etype, uri, extid = self.system_source.eid_type_source(session, |
|
813 |
eid) |
|
814 |
finally: |
|
815 |
if reset_pool: |
|
816 |
session.reset_pool() |
|
817 |
self._type_source_cache[eid] = (etype, uri, extid) |
|
818 |
if uri != 'system': |
|
819 |
self._extid_cache[(extid, uri)] = eid |
|
820 |
return etype, uri, extid |
|
821 |
||
822 |
def clear_caches(self, eids): |
|
823 |
etcache = self._type_source_cache |
|
824 |
extidcache = self._extid_cache |
|
825 |
rqlcache = self.querier._rql_cache |
|
826 |
for eid in eids: |
|
827 |
try: |
|
828 |
etype, uri, extid = etcache.pop(typed_eid(eid)) # may be a string in some cases |
|
829 |
rqlcache.pop('%s X WHERE X eid %s' % (etype, eid), None) |
|
830 |
extidcache.pop((extid, uri), None) |
|
831 |
except KeyError: |
|
832 |
etype = None |
|
833 |
rqlcache.pop('Any X WHERE X eid %s' % eid, None) |
|
834 |
for source in self.sources: |
|
835 |
source.clear_eid_cache(eid, etype) |
|
1482 | 836 |
|
0 | 837 |
def type_from_eid(self, eid, session=None): |
838 |
"""return the type of the entity with id <eid>""" |
|
839 |
return self.type_and_source_from_eid(eid, session)[0] |
|
1482 | 840 |
|
0 | 841 |
def source_from_eid(self, eid, session=None): |
842 |
"""return the source for the given entity's eid""" |
|
843 |
return self.sources_by_uri[self.type_and_source_from_eid(eid, session)[1]] |
|
1482 | 844 |
|
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
845 |
def querier_cache_key(self, session, rql, args, eidkeys): |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
846 |
cachekey = [rql] |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
847 |
for key in sorted(eidkeys): |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
848 |
try: |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
849 |
etype = self.type_from_eid(args[key], session) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
850 |
except KeyError: |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
851 |
raise QueryError('bad cache key %s (no value)' % key) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
852 |
except TypeError: |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
853 |
raise QueryError('bad cache key %s (value: %r)' % ( |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
854 |
key, args[key])) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
855 |
cachekey.append(etype) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
856 |
# ensure eid is correctly typed in args |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
857 |
args[key] = typed_eid(args[key]) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
858 |
return tuple(cachekey) |
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
859 |
|
0 | 860 |
def eid2extid(self, source, eid, session=None): |
861 |
"""get local id from an eid""" |
|
862 |
etype, uri, extid = self.type_and_source_from_eid(eid, session) |
|
863 |
if source.uri != uri: |
|
864 |
# eid not from the given source |
|
865 |
raise UnknownEid(eid) |
|
866 |
return extid |
|
867 |
||
1954 | 868 |
def extid2eid(self, source, extid, etype, session=None, insert=True, |
1250
5c20a7f13c84
new recreate argument to extid2eid when an external source want to recreate entities previously imported with a predictable ext id
sylvain.thenault@logilab.fr
parents:
1228
diff
changeset
|
869 |
recreate=False): |
0 | 870 |
"""get eid from a local id. An eid is attributed if no record is found""" |
1954 | 871 |
cachekey = (extid, source.uri) |
0 | 872 |
try: |
873 |
return self._extid_cache[cachekey] |
|
874 |
except KeyError: |
|
875 |
pass |
|
876 |
reset_pool = False |
|
877 |
if session is None: |
|
878 |
session = self.internal_session() |
|
879 |
reset_pool = True |
|
1954 | 880 |
eid = self.system_source.extid2eid(session, source, extid) |
0 | 881 |
if eid is not None: |
882 |
self._extid_cache[cachekey] = eid |
|
1954 | 883 |
self._type_source_cache[eid] = (etype, source.uri, extid) |
5398
b9e1abe1bdfe
[repo] cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5385
diff
changeset
|
884 |
# XXX used with extlite (eg vcsfile), probably not needed anymore |
1250
5c20a7f13c84
new recreate argument to extid2eid when an external source want to recreate entities previously imported with a predictable ext id
sylvain.thenault@logilab.fr
parents:
1228
diff
changeset
|
885 |
if recreate: |
1954 | 886 |
entity = source.before_entity_insertion(session, extid, etype, eid) |
1250
5c20a7f13c84
new recreate argument to extid2eid when an external source want to recreate entities previously imported with a predictable ext id
sylvain.thenault@logilab.fr
parents:
1228
diff
changeset
|
887 |
entity._cw_recreating = True |
5c20a7f13c84
new recreate argument to extid2eid when an external source want to recreate entities previously imported with a predictable ext id
sylvain.thenault@logilab.fr
parents:
1228
diff
changeset
|
888 |
if source.should_call_hooks: |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
889 |
self.hm.call_hooks('before_add_entity', session, entity=entity) |
1250
5c20a7f13c84
new recreate argument to extid2eid when an external source want to recreate entities previously imported with a predictable ext id
sylvain.thenault@logilab.fr
parents:
1228
diff
changeset
|
890 |
# XXX add fti op ? |
1954 | 891 |
source.after_entity_insertion(session, extid, entity) |
1250
5c20a7f13c84
new recreate argument to extid2eid when an external source want to recreate entities previously imported with a predictable ext id
sylvain.thenault@logilab.fr
parents:
1228
diff
changeset
|
892 |
if source.should_call_hooks: |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
893 |
self.hm.call_hooks('after_add_entity', session, entity=entity) |
0 | 894 |
if reset_pool: |
895 |
session.reset_pool() |
|
896 |
return eid |
|
897 |
if not insert: |
|
898 |
return |
|
1954 | 899 |
# no link between extid and eid, create one using an internal session |
0 | 900 |
# since the current session user may not have required permissions to |
901 |
# do necessary stuff and we don't want to commit user session. |
|
902 |
# |
|
450
5e14ea0e81c8
a note for later
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
341
diff
changeset
|
903 |
# Moreover, even if session is already an internal session but is |
0 | 904 |
# processing a commit, we have to use another one |
905 |
if not session.is_internal_session: |
|
906 |
session = self.internal_session() |
|
907 |
reset_pool = True |
|
908 |
try: |
|
909 |
eid = self.system_source.create_eid(session) |
|
910 |
self._extid_cache[cachekey] = eid |
|
1954 | 911 |
self._type_source_cache[eid] = (etype, source.uri, extid) |
912 |
entity = source.before_entity_insertion(session, extid, etype, eid) |
|
5398
b9e1abe1bdfe
[repo] cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5385
diff
changeset
|
913 |
entity.edited_attributes = set(entity) |
0 | 914 |
if source.should_call_hooks: |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
915 |
self.hm.call_hooks('before_add_entity', session, entity=entity) |
450
5e14ea0e81c8
a note for later
Aurelien Campeas <aurelien.campeas@logilab.fr>
parents:
341
diff
changeset
|
916 |
# XXX call add_info with complete=False ? |
1954 | 917 |
self.add_info(session, entity, source, extid) |
918 |
source.after_entity_insertion(session, extid, entity) |
|
0 | 919 |
if source.should_call_hooks: |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
920 |
self.hm.call_hooks('after_add_entity', session, entity=entity) |
0 | 921 |
else: |
922 |
# minimal meta-data |
|
923 |
session.execute('SET X is E WHERE X eid %(x)s, E name %(name)s', |
|
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
924 |
{'x': entity.eid, 'name': entity.__regid__}) |
0 | 925 |
session.commit(reset_pool) |
926 |
return eid |
|
927 |
except: |
|
928 |
session.rollback(reset_pool) |
|
929 |
raise |
|
1482 | 930 |
|
0 | 931 |
def add_info(self, session, entity, source, extid=None, complete=True): |
932 |
"""add type and source info for an eid into the system table, |
|
933 |
and index the entity with the full text index |
|
934 |
""" |
|
935 |
# begin by inserting eid/type/source/extid into the entities table |
|
5067
adc2122eed03
[repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5043
diff
changeset
|
936 |
hook.set_operation(session, 'neweids', entity.eid, |
adc2122eed03
[repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5043
diff
changeset
|
937 |
hook.CleanupNewEidsCacheOp) |
4806
4f12f59b1a13
[fti] refactor and fix full text indexation handling
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4756
diff
changeset
|
938 |
self.system_source.add_info(session, entity, source, extid, complete) |
1482 | 939 |
|
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
940 |
def delete_info(self, session, entity, sourceuri, extid): |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
941 |
"""called by external source when some entity known by the system source |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
942 |
has been deleted in the external source |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
943 |
""" |
5067
adc2122eed03
[repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5043
diff
changeset
|
944 |
# mark eid as being deleted in session info and setup cache update |
adc2122eed03
[repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5043
diff
changeset
|
945 |
# operation |
adc2122eed03
[repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5043
diff
changeset
|
946 |
hook.set_operation(session, 'pendingeids', entity.eid, |
adc2122eed03
[repo] more efficient eid cache operations handling based on set_operation; refactor
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5043
diff
changeset
|
947 |
hook.CleanupDeletedEidsCacheOp) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
948 |
self._delete_info(session, entity, sourceuri, extid) |
1482 | 949 |
|
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
950 |
def _delete_info(self, session, entity, sourceuri, extid): |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
951 |
# attributes=None, relations=None): |
0 | 952 |
"""delete system information on deletion of an entity: |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
953 |
* delete all remaining relations from/to this entity |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
954 |
* call delete info on the system source which will transfer record from |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
955 |
the entities table to the deleted_entities table |
0 | 956 |
""" |
2641
9c33d98a074e
R [schema hooks] big refactoring / reorganization for clearer code, a few fixes on the way
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2631
diff
changeset
|
957 |
pendingrtypes = session.transaction_data.get('pendingrtypes', ()) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
958 |
# delete remaining relations: if user can delete the entity, he can |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
959 |
# delete all its relations without security checking |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
960 |
with security_enabled(session, read=False, write=False): |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
961 |
eid = entity.eid |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
962 |
for rschema, _, role in entity.e_schema.relation_definitions(): |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
963 |
rtype = rschema.type |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
964 |
if rtype in schema.VIRTUAL_RTYPES or rtype in pendingrtypes: |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
965 |
continue |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
966 |
if role == 'subject': |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
967 |
# don't skip inlined relation so they are regularly |
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
968 |
# deleted and so hooks are correctly called |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
969 |
rql = 'DELETE X %s Y WHERE X eid %%(x)s' % rtype |
4835
13b0b96d7982
[repo] enhanced security handling: deprecates unsafe_execute, in favor of explicit read/write security control using the `enabled_security` context manager. Also code executed on the repository side is now unsafe by default.
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4834
diff
changeset
|
970 |
else: |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
971 |
rql = 'DELETE Y %s X WHERE X eid %%(x)s' % rtype |
5174
78438ad513ca
#759035: Automate addition of eid cachekey in RQL analysis
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5115
diff
changeset
|
972 |
session.execute(rql, {'x': eid}, build_descr=False) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
973 |
self.system_source.delete_info(session, entity, sourceuri, extid) |
0 | 974 |
|
975 |
def locate_relation_source(self, session, subject, rtype, object): |
|
976 |
subjsource = self.source_from_eid(subject, session) |
|
977 |
objsource = self.source_from_eid(object, session) |
|
3042
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
978 |
if not subjsource is objsource: |
0 | 979 |
source = self.system_source |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
980 |
if not (subjsource.may_cross_relation(rtype) |
3042
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
981 |
and objsource.may_cross_relation(rtype)): |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
982 |
raise MultiSourcesError( |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
983 |
"relation %s can't be crossed among sources" |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
984 |
% rtype) |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
985 |
elif not subjsource.support_relation(rtype): |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
986 |
source = self.system_source |
0 | 987 |
else: |
988 |
source = subjsource |
|
3042
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
989 |
if not source.support_relation(rtype, True): |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
990 |
raise MultiSourcesError( |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
991 |
"source %s doesn't support write of %s relation" |
d2455badf7fb
[multi-sources] enhance relation'source detection to avoid inconsistency
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2967
diff
changeset
|
992 |
% (source.uri, rtype)) |
0 | 993 |
return source |
1482 | 994 |
|
0 | 995 |
def locate_etype_source(self, etype): |
996 |
for source in self.sources: |
|
997 |
if source.support_entity(etype, 1): |
|
998 |
return source |
|
999 |
else: |
|
1000 |
raise ETypeNotSupportedBySources(etype) |
|
1482 | 1001 |
|
5068
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1002 |
def init_entity_caches(self, session, entity, source): |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1003 |
"""add entity to session entities cache and repo's extid cache. |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1004 |
Return entity's ext id if the source isn't the system source. |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1005 |
""" |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1006 |
session.set_entity_cache(entity) |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1007 |
suri = source.uri |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1008 |
if suri == 'system': |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1009 |
extid = None |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1010 |
else: |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1011 |
extid = source.get_extid(entity) |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1012 |
self._extid_cache[(str(extid), suri)] = entity.eid |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1013 |
self._type_source_cache[entity.eid] = (entity.__regid__, suri, extid) |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1014 |
return extid |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1015 |
|
0 | 1016 |
def glob_add_entity(self, session, entity): |
1017 |
"""add an entity to the repository |
|
1482 | 1018 |
|
0 | 1019 |
the entity eid should originaly be None and a unique eid is assigned to |
1020 |
the entity instance |
|
1021 |
""" |
|
3584
585c4423a54c
set edited_attributes before call to pre_add_hook
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3567
diff
changeset
|
1022 |
# init edited_attributes before calling before_add_entity hooks |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1023 |
entity._cw_is_saved = False # entity has an eid but is not yet saved |
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1024 |
entity.edited_attributes = set(entity) # XXX cw_edited_attributes |
0 | 1025 |
eschema = entity.e_schema |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1026 |
source = self.locate_etype_source(entity.__regid__) |
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1027 |
# allocate an eid to the entity before calling hooks |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1028 |
entity.eid = self.system_source.create_eid(session) |
5068
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1029 |
# set caches asap |
10c3422d7419
[repo] on add entity, set cache as soon as possible + fill type/source cache
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5067
diff
changeset
|
1030 |
extid = self.init_entity_caches(session, entity, source) |
2600
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1031 |
if server.DEBUG & server.DBG_REPO: |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1032 |
print 'ADD entity', entity.__regid__, entity.eid, dict(entity) |
0 | 1033 |
relations = [] |
2929
51cdfe069e10
fix edited_attributes handling when adding entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2875
diff
changeset
|
1034 |
if source.should_call_hooks: |
2968
0e3460341023
somewhat painful backport of 3.5 branch, should mostly be ok
Sylvain Thénault <sylvain.thenault@logilab.fr>
diff
changeset
|
1035 |
self.hm.call_hooks('before_add_entity', session, entity=entity) |
2929
51cdfe069e10
fix edited_attributes handling when adding entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2875
diff
changeset
|
1036 |
# XXX use entity.keys here since edited_attributes is not updated for |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1037 |
# inline relations XXX not true, right? (see edited_attributes |
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1038 |
# affectation above) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1039 |
for attr in entity.iterkeys(): |
3689
deb13e88e037
follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3606
diff
changeset
|
1040 |
rschema = eschema.subjrels[attr] |
deb13e88e037
follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3606
diff
changeset
|
1041 |
if not rschema.final: # inlined relation |
0 | 1042 |
relations.append((attr, entity[attr])) |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1043 |
entity._cw_set_defaults() |
4843
5f7363416765
fix hooks control method name + other litle cleanups
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4835
diff
changeset
|
1044 |
if session.is_hook_category_activated('integrity'): |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1045 |
entity._cw_check(creation=True) |
0 | 1046 |
source.add_entity(session, entity) |
1047 |
self.add_info(session, entity, source, extid, complete=False) |
|
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1048 |
entity._cw_is_saved = True # entity has an eid and is saved |
2647
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1049 |
# prefill entity relation caches |
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1050 |
for rschema in eschema.subject_relations(): |
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1051 |
rtype = str(rschema) |
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
1052 |
if rtype in schema.VIRTUAL_RTYPES: |
2647
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1053 |
continue |
3689
deb13e88e037
follow yams 0.25 api changes to improve performance
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3606
diff
changeset
|
1054 |
if rschema.final: |
2647
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1055 |
entity.setdefault(rtype, None) |
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1056 |
else: |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1057 |
entity.cw_set_relation_cache(rtype, 'subject', |
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1058 |
session.empty_rset()) |
2647
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1059 |
for rschema in eschema.object_relations(): |
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1060 |
rtype = str(rschema) |
2839
6419af16faa0
imports cleanup
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2835
diff
changeset
|
1061 |
if rtype in schema.VIRTUAL_RTYPES: |
2647
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1062 |
continue |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1063 |
entity.cw_set_relation_cache(rtype, 'object', session.empty_rset()) |
2756
2c94606eefc0
[server caching] should set cache before call to after_add_entity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2731
diff
changeset
|
1064 |
# set inline relation cache before call to after_add_entity |
2c94606eefc0
[server caching] should set cache before call to after_add_entity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2731
diff
changeset
|
1065 |
for attr, value in relations: |
2c94606eefc0
[server caching] should set cache before call to after_add_entity hooks
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2731
diff
changeset
|
1066 |
session.update_rel_cache_add(entity.eid, attr, value) |
0 | 1067 |
# trigger after_add_entity after after_add_relation |
1068 |
if source.should_call_hooks: |
|
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1069 |
self.hm.call_hooks('after_add_entity', session, entity=entity) |
0 | 1070 |
# call hooks for inlined relations |
1071 |
for attr, value in relations: |
|
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1072 |
self.hm.call_hooks('before_add_relation', session, |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1073 |
eidfrom=entity.eid, rtype=attr, eidto=value) |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1074 |
self.hm.call_hooks('after_add_relation', session, |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1075 |
eidfrom=entity.eid, rtype=attr, eidto=value) |
0 | 1076 |
return entity.eid |
1482 | 1077 |
|
2667
c8aa82538d8e
[repo] explicitly specifies edited attributes to glob_update_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2665
diff
changeset
|
1078 |
def glob_update_entity(self, session, entity, edited_attributes): |
0 | 1079 |
"""replace an entity in the repository |
1080 |
the type and the eid of an entity must not be changed |
|
1081 |
""" |
|
2600
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1082 |
if server.DEBUG & server.DBG_REPO: |
5072
072ae171aeb0
[cleanup] style fixes, add nodes, 0.2 cents refactorings
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5068
diff
changeset
|
1083 |
print 'UPDATE entity', entity.__regid__, entity.eid, \ |
2667
c8aa82538d8e
[repo] explicitly specifies edited attributes to glob_update_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2665
diff
changeset
|
1084 |
dict(entity), edited_attributes |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1085 |
hm = self.hm |
0 | 1086 |
eschema = entity.e_schema |
2647
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1087 |
session.set_entity_cache(entity) |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1088 |
orig_edited_attributes = getattr(entity, 'edited_attributes', None) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1089 |
entity.edited_attributes = edited_attributes |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1090 |
try: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1091 |
if session.is_hook_category_activated('integrity'): |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1092 |
entity._cw_check() |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1093 |
only_inline_rels, need_fti_update = True, False |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1094 |
relations = [] |
5292
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1095 |
source = self.source_from_eid(entity.eid, session) |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1096 |
for attr in list(edited_attributes): |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1097 |
if attr == 'eid': |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1098 |
continue |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1099 |
rschema = eschema.subjrels[attr] |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1100 |
if rschema.final: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1101 |
if getattr(eschema.rdef(attr), 'fulltextindexed', False): |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1102 |
need_fti_update = True |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1103 |
only_inline_rels = False |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1104 |
else: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1105 |
# inlined relation |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1106 |
previous_value = entity.related(attr) or None |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1107 |
if previous_value is not None: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1108 |
previous_value = previous_value[0][0] # got a result set |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1109 |
if previous_value == entity[attr]: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1110 |
previous_value = None |
5292
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1111 |
elif source.should_call_hooks: |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1112 |
hm.call_hooks('before_delete_relation', session, |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1113 |
eidfrom=entity.eid, rtype=attr, |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1114 |
eidto=previous_value) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1115 |
relations.append((attr, entity[attr], previous_value)) |
5292
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1116 |
if source.should_call_hooks: |
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1117 |
# call hooks for inlined relations |
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1118 |
for attr, value, _ in relations: |
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1119 |
hm.call_hooks('before_add_relation', session, |
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1120 |
eidfrom=entity.eid, rtype=attr, eidto=value) |
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1121 |
if not only_inline_rels: |
8b496574b65c
[repo] fix more indentation pb in repo.glob_update_entity, causing duplicated inline relation hooks call
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5187
diff
changeset
|
1122 |
hm.call_hooks('before_update_entity', session, entity=entity) |
5187
5f9a2b32c9e1
[repository] fix bad indentation in glob_update_entity, source.update_entity() should be called once per update
Adrien Di Mascio <Adrien.DiMascio@logilab.fr>
parents:
5167
diff
changeset
|
1123 |
source.update_entity(session, entity) |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1124 |
self.system_source.update_info(session, entity, need_fti_update) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1125 |
if source.should_call_hooks: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1126 |
if not only_inline_rels: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1127 |
hm.call_hooks('after_update_entity', session, entity=entity) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1128 |
for attr, value, prevvalue in relations: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1129 |
# if the relation is already cached, update existant cache |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1130 |
relcache = entity.cw_relation_cached(attr, 'subject') |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1131 |
if prevvalue is not None: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1132 |
hm.call_hooks('after_delete_relation', session, |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1133 |
eidfrom=entity.eid, rtype=attr, eidto=prevvalue) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1134 |
if relcache is not None: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1135 |
session.update_rel_cache_del(entity.eid, attr, prevvalue) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1136 |
del_existing_rel_if_needed(session, entity.eid, attr, value) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1137 |
if relcache is not None: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1138 |
session.update_rel_cache_add(entity.eid, attr, value) |
2647
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1139 |
else: |
5557
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1140 |
entity.cw_set_relation_cache(attr, 'subject', |
1a534c596bff
[entity] continue cleanup of Entity/AnyEntity namespace
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5556
diff
changeset
|
1141 |
session.eid_rset(value)) |
5115
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1142 |
hm.call_hooks('after_add_relation', session, |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1143 |
eidfrom=entity.eid, rtype=attr, eidto=value) |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1144 |
finally: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1145 |
if orig_edited_attributes is not None: |
2e43ef618d14
[repository] forbid usage of set_attributes() in before_add_entity
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
5093
diff
changeset
|
1146 |
entity.edited_attributes = orig_edited_attributes |
0 | 1147 |
|
1148 |
def glob_delete_entity(self, session, eid): |
|
1149 |
"""delete an entity and all related entities from the repository""" |
|
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1150 |
entity = session.entity_from_eid(eid) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1151 |
etype, sourceuri, extid = self.type_and_source_from_eid(eid, session) |
2600
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1152 |
if server.DEBUG & server.DBG_REPO: |
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1153 |
print 'DELETE entity', etype, eid |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1154 |
source = self.sources_by_uri[sourceuri] |
0 | 1155 |
if source.should_call_hooks: |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1156 |
self.hm.call_hooks('before_delete_entity', session, entity=entity) |
4913
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1157 |
self._delete_info(session, entity, sourceuri, extid) |
083b4d454192
server/web api for accessing to deleted_entites
Katia Saurfelt <katia.saurfelt@logilab.fr>
parents:
4899
diff
changeset
|
1158 |
source.delete_entity(session, entity) |
0 | 1159 |
if source.should_call_hooks: |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1160 |
self.hm.call_hooks('after_delete_entity', session, entity=entity) |
0 | 1161 |
# don't clear cache here this is done in a hook on commit |
1482 | 1162 |
|
0 | 1163 |
def glob_add_relation(self, session, subject, rtype, object): |
1164 |
"""add a relation to the repository""" |
|
2600
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1165 |
if server.DEBUG & server.DBG_REPO: |
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1166 |
print 'ADD relation', subject, rtype, object |
0 | 1167 |
source = self.locate_relation_source(session, subject, rtype, object) |
1168 |
if source.should_call_hooks: |
|
1169 |
del_existing_rel_if_needed(session, subject, rtype, object) |
|
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1170 |
self.hm.call_hooks('before_add_relation', session, |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1171 |
eidfrom=subject, rtype=rtype, eidto=object) |
0 | 1172 |
source.add_relation(session, subject, rtype, object) |
2647
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1173 |
rschema = self.schema.rschema(rtype) |
4467
0e73d299730a
fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4252
diff
changeset
|
1174 |
session.update_rel_cache_add(subject, rtype, object, rschema.symmetric) |
0 | 1175 |
if source.should_call_hooks: |
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1176 |
self.hm.call_hooks('after_add_relation', session, |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1177 |
eidfrom=subject, rtype=rtype, eidto=object) |
0 | 1178 |
|
1179 |
def glob_delete_relation(self, session, subject, rtype, object): |
|
1180 |
"""delete a relation from the repository""" |
|
2600
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1181 |
if server.DEBUG & server.DBG_REPO: |
6cd6c5d11b45
[F repo debugging] log repo event on DBG_REPO debug level
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2596
diff
changeset
|
1182 |
print 'DELETE relation', subject, rtype, object |
0 | 1183 |
source = self.locate_relation_source(session, subject, rtype, object) |
1184 |
if source.should_call_hooks: |
|
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1185 |
self.hm.call_hooks('before_delete_relation', session, |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1186 |
eidfrom=subject, rtype=rtype, eidto=object) |
0 | 1187 |
source.delete_relation(session, subject, rtype, object) |
2647
b0a2e779845c
enable server side entity caching, 25% speedup on codenaf insertion. ALL CW TESTS OK
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2641
diff
changeset
|
1188 |
rschema = self.schema.rschema(rtype) |
4467
0e73d299730a
fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4252
diff
changeset
|
1189 |
session.update_rel_cache_del(subject, rtype, object, rschema.symmetric) |
0e73d299730a
fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4252
diff
changeset
|
1190 |
if rschema.symmetric: |
0e73d299730a
fix long-waiting symetric typo: should be spelled symmetric. Add auto database migration on schema deserialization
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
4252
diff
changeset
|
1191 |
# on symmetric relation, we can't now in which sense it's |
0 | 1192 |
# stored so try to delete both |
1193 |
source.delete_relation(session, object, rtype, subject) |
|
1194 |
if source.should_call_hooks: |
|
2835
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1195 |
self.hm.call_hooks('after_delete_relation', session, |
04034421b072
[hooks] major refactoring:
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2822
diff
changeset
|
1196 |
eidfrom=subject, rtype=rtype, eidto=object) |
0 | 1197 |
|
1198 |
||
1199 |
# pyro handling ########################################################### |
|
1482 | 1200 |
|
0 | 1201 |
def pyro_register(self, host=''): |
1202 |
"""register the repository as a pyro object""" |
|
3606
8326aceecb46
fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3584
diff
changeset
|
1203 |
import tempfile |
8326aceecb46
fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3584
diff
changeset
|
1204 |
from logilab.common.pyro_ext import register_object, config |
8326aceecb46
fix startup as a daemon
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3584
diff
changeset
|
1205 |
config.PYRO_STORAGE = tempfile.gettempdir() # XXX until lgc > 0.45.1 is out |
3539
f3b14d052798
[pyro] merge pyro-id / pyro-instance-id options, put all pyro options in the same section of the configuration file
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3250
diff
changeset
|
1206 |
appid = self.config['pyro-instance-id'] or self.config.appid |
2665
0c6281487f90
[pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2650
diff
changeset
|
1207 |
daemon = register_object(self, appid, self.config['pyro-ns-group'], |
0c6281487f90
[pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2650
diff
changeset
|
1208 |
self.config['pyro-host'], |
0c6281487f90
[pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2650
diff
changeset
|
1209 |
self.config['pyro-ns-host']) |
0 | 1210 |
msg = 'repository registered as a pyro object using group %s and id %s' |
2665
0c6281487f90
[pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2650
diff
changeset
|
1211 |
self.info(msg, self.config['pyro-ns-group'], appid) |
0 | 1212 |
self.pyro_registered = True |
1213 |
return daemon |
|
1482 | 1214 |
|
1228
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1215 |
# multi-sources planner helpers ########################################### |
1482 | 1216 |
|
1228
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1217 |
@cached |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1218 |
def rel_type_sources(self, rtype): |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1219 |
return [source for source in self.sources |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1220 |
if source.support_relation(rtype) |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1221 |
or rtype in source.dont_cross_relations] |
1482 | 1222 |
|
1228
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1223 |
@cached |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1224 |
def can_cross_relation(self, rtype): |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1225 |
return [source for source in self.sources |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1226 |
if source.support_relation(rtype) |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1227 |
and rtype in source.cross_relations] |
1482 | 1228 |
|
1228
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1229 |
@cached |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1230 |
def is_multi_sources_relation(self, rtype): |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1231 |
return any(source for source in self.sources |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1232 |
if not source is self.system_source |
91ae10ffb611
* refactor ms planner (renaming, reorganization)
sylvain.thenault@logilab.fr
parents:
1217
diff
changeset
|
1233 |
and source.support_relation(rtype)) |
1482 | 1234 |
|
0 | 1235 |
|
1236 |
def pyro_unregister(config): |
|
1237 |
"""unregister the repository from the pyro name server""" |
|
2665
0c6281487f90
[pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2650
diff
changeset
|
1238 |
from logilab.common.pyro_ext import ns_unregister |
3539
f3b14d052798
[pyro] merge pyro-id / pyro-instance-id options, put all pyro options in the same section of the configuration file
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
3250
diff
changeset
|
1239 |
appid = config['pyro-instance-id'] or config.appid |
2665
0c6281487f90
[pyro] use lgc.pyro_ext, simplify pyro related options
Sylvain Thénault <sylvain.thenault@logilab.fr>
parents:
2650
diff
changeset
|
1240 |
ns_unregister(appid, config['pyro-ns-group'], config['pyro-ns-host']) |
0 | 1241 |
|
1242 |
||
1243 |
from logging import getLogger |
|
1244 |
from cubicweb import set_log_methods |
|
1245 |
set_log_methods(Repository, getLogger('cubicweb.repository')) |